-- Logs begin at Wed 2022-06-29 07:19:48 UTC, end at Wed 2022-06-29 08:26:54 UTC. -- Jun 29 07:47:40.041619 dv-cinder-768105-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 29 07:47:40.049905 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 29 07:47:40.051945 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 29 07:47:40 2022] *** Jun 29 07:47:40.051945 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 29 07:47:40.051945 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 29 07:47:40.051945 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: nodename: dv-cinder-768105-14-iscsi Jun 29 07:47:40.051945 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: machine: x86_64 Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: clock source: unix Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: pcre jit disabled Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: detected number of CPU cores: 4 Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: current working directory: / Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: detected binary path: /usr/bin/uwsgi-core Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: your processes number limit is 47680 Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: your memory page size is 4096 bytes Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: detected max file descriptor number: 2048 Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: lock engine: pthread robust mutexes Jun 29 07:47:40.052266 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: thunder lock: enabled Jun 29 07:47:40.052815 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 29 07:47:40.052815 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 29 07:47:40.098899 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: Python main interpreter initialized at 0x5627c6d2b9c0 Jun 29 07:47:40.098899 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: python threads support enabled Jun 29 07:47:40.098899 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: your server socket listen backlog is limited to 100 connections Jun 29 07:47:40.098899 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: your mercy for graceful operations on workers is 90 seconds Jun 29 07:47:40.099294 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: mapped 403077 bytes (393 KB) for 2 cores Jun 29 07:47:40.099369 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: *** Operational MODE: preforking *** Jun 29 07:47:40.099685 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: *** uWSGI is running in multiple interpreter mode *** Jun 29 07:47:40.099685 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: spawned uWSGI master process (pid: 80098) Jun 29 07:47:40.099902 dv-cinder-768105-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 29 07:47:40.101421 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: spawned uWSGI worker 1 (pid: 80107, cores: 1) Jun 29 07:47:40.101421 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: spawned uWSGI worker 2 (pid: 80108, cores: 1) Jun 29 07:47:40.101421 dv-cinder-768105-14-iscsi devstack@keystone.service[80098]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 29 07:47:45.637728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 07:47:45.637728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: from cryptography.utils import int_from_bytes Jun 29 07:47:45.638513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 07:47:45.638513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: from cryptography.utils import int_from_bytes Jun 29 07:47:45.638624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 07:47:45.638624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: from cryptography.utils import int_from_bytes Jun 29 07:47:45.639030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 07:47:45.639030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: from cryptography.utils import int_from_bytes Jun 29 07:47:45.689363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 07:47:45.689564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 07:47:45.689728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] command line args: [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 07:47:45.689996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 07:47:45.690087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 07:47:45.690170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ================================================================================ {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 07:47:45.690247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 07:47:45.690398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] command line args: [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 07:47:45.690482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] admin_token = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.690649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] config_dir = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.690739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 07:47:45.690817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ================================================================================ {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 07:47:45.690894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] config_source = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] admin_token = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] debug = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] config_dir = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] config_source = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] debug = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.691955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] list_limit = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] log_config_append = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.692918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] list_limit = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] log_dir = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] log_config_append = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] log_file = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] log_dir = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] log_file = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.693957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.694929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.695989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.696144 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.696263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.696391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.696481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.696639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.696723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.696882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.697213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.697333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.697465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] publish_errors = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.697635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.697844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.697971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.698096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.698210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] publish_errors = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.698390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.698513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.698652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.698810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.698908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] standard_threads = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] use_journal = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.699972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] standard_threads = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] use_json = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] use_stderr = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] use_syslog = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.700778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] use_journal = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.701023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.701151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] use_json = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.701273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.701393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] use_stderr = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.701393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] use_syslog = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.701618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.701618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.701837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 07:47:45.701950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.702995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.703939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.704874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.705905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.706939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.707968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.708907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.709998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.710128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.710248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.710367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.710481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.710595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.710708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.710896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.711927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.712921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.713783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.714379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.714497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.714631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.714758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.714758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.714974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.715901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.716963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.717908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.718977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.719915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720528 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.720879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.721983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.722973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.723931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.724969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.725914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] auth.external = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.726970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] auth.password = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] auth.token = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] auth.external = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.727987 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] auth.password = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] auth.token = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.728940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.729083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.729211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] credential.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.729328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.729444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.729567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.730274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.730392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.730509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.730624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.730747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.730872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.730990 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] credential.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.731925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.732867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.733946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734066 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.734898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.735957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.736919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.737997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.738998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.739902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.740812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.741939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742766 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.742890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.743891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.744829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.745110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.745210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.745324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.745439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.745553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.745680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.745887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.746954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.747068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.747182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.747416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.747536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.747650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.747763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.747877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.748100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.748281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.748404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.748518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.748632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.748751 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.748870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.749930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.750045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.750310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.750429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.750545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.750759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.750883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.750998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.751113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.751227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.751349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.751493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.751618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.751782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.751901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.752016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.752139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.752256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.752496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.752704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.752824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.752971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.753126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.753272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.753393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.753509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.753623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.753775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.753904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.754021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.754246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.754367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.754482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.754597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.754751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.754871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.755050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.755170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.755321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.755440 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.755563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.755681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.755836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.756096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.756215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.756351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.756478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.756641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.756762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.756878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.757075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.757197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.757363 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.757491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.757607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.757724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.757906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.758032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.758201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.758394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.758514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.758714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.758835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.759009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.759137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.759293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.759438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.759585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.759706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.759823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.760030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.760151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.760361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] role.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.760479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.760615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] role.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.760735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.760942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.761179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.761331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.761450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.761566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.761682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.761812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.761935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.762050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.762247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.762366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.762500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.762698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.762818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.762935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.763985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.764091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.764234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.764355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.764585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.764704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.764842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.764970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.765283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.765403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.765538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.765657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.765796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.765929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.766078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.766200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.766315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.766414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.766558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.766688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] role.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.766851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] role.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.767973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.768121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.768276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.768405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.768710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.768830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.769153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.769288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.769406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.769536 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.769654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.769769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.769884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.770025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.770145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.770316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.770437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.770552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.770804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.770923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.771038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.771254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.771376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.771534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.771654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.771810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.771931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.772055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.772340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.772522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.772522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.772742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.772742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.772968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.773114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.773294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.773418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.773615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.773733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.773863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.773982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.774202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.774320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.774470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.connection = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.774602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.774713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.774839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.774956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.775073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.775224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.775344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.775459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.775682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.775804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.775919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.776035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.776149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.776264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.776959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.777116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.777387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.777508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.777701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.777819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.778054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.778181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.778299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.778415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.778651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.778773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.778898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779517 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.779930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.780045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 07:47:45.780181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.780424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.780674 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.781066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.781346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.781710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.781955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.782309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.782564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.782843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.783196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.783560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.783983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.784253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.connection = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.784734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.785026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.785276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.785635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.785984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.786324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.786687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.786933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.787304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.787572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.787933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.788175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.788546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.788813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.789199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.789435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.789795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.790039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 07:47:45.790238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80107) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 07:47:45.814257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.814519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.814799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.824479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.825993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.826230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.826461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.834876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.853259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.853389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.853543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:45.853627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.003685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.003865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.056890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.057140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.057230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.057308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.123471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.123696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.123919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.124795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.125036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.150941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.151025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.151155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.151237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.151399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.151540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.162777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.163012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.176609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.176846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.177966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80107) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.178046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.178177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.178388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80108) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 07:47:46.195499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.195708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 07:47:46.196047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.196146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.196246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 07:47:46.196327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 07:47:46.196630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.196752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.196834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 07:47:46.197032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.197117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.197289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.197372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.197541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.197634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.197844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.209036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.209313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.209397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.209578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.209670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.209839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.209918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.210123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.218169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.218467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.221469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.221746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.221829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.222025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.222121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.222303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.222385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.222579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.222661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.222836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.222929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223363 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.223971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.224152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.224243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.224414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.251120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.251739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.252729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.253029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.253392 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.253670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.258301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.258533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.258809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.258925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.259201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.259600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.266807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.267084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.267513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.267800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.271562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.271844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.272120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.272270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.272399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.272559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.272670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.272840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.272941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.273148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.273231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.273422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.273502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.273695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.273963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.274231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.291963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.292764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.293535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.293811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.297268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.299344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.301848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.302143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.303442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.303740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.307302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.309606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.311478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.313203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.315118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.315416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.315697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.315979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.316274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.316557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.316834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.317162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.317445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.317730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.318020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.318315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.335663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.335946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.336214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.336474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.337611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.337901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.338168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.338429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.345004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.346704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.346856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.346978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.348584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.348858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.350178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.350459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.350736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.351013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.351295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.352058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.352339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.352616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.352910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.353210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.365450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.365744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.366026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.366303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.366584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.366852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.367128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.367431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.367732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.368023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.368304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.368584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.370461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.370816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.387058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.389138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.390487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.392537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.393871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.395485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.395921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.397298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.397579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.397699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.397859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.399326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.399606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.399894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.405735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.407327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.407716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.409333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.410589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.410879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.411153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.411426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.412558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.412846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.413261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.413538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.420835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.421143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.421433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.422959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.423243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.423524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.429496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80107) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.429773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.430052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.430329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.430597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.430867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.431153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.431330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80108) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 07:47:46.431414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.431612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.431785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.431890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.432066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.432150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.432344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.432435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.432625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.433324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.433409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.433660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80107) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.433823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.434213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.434497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.434777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.435065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.435332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.435608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.435874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80108) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 07:47:46.471760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5627c6d2b9c0 pid: 80107 (default app) Jun 29 07:47:46.474100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5627c6d2b9c0 pid: 80108 (default app) Jun 29 07:47:46.475249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12c427-c6cc-4660-b9df-092f8246d855 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:46.475539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12c427-c6cc-4660-b9df-092f8246d855 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:46.475819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12c427-c6cc-4660-b9df-092f8246d855 None None] PATH_INFO: `/v3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:46.480604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Wed Jun 29 07:47:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:47:54.281392 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef628b78-7123-41fa-9d73-d82e99a65691 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:54.281980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef628b78-7123-41fa-9d73-d82e99a65691 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:54.281980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef628b78-7123-41fa-9d73-d82e99a65691 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:54.286269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:47:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:47:54.291480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:54.292360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:54.292557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:54.362681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: INFO dbcounter [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] Registered counter for database keystone Jun 29 07:47:54.384003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG oslo_db.sqlalchemy.engines [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80107) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 07:47:54.390651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writer thread running {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 07:47:54.568429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG passlib.handlers.bcrypt [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80107) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 07:47:54.568849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG passlib.handlers.bcrypt [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80107) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 07:47:54.623298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:47:54.624897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:47:54.746552 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:47:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 29 07:47:54.905735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef628b78-7123-41fa-9d73-d82e99a65691 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:47:54.977007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: INFO dbcounter [None req-ef628b78-7123-41fa-9d73-d82e99a65691 None None] Registered counter for database keystone Jun 29 07:47:54.997024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG oslo_db.sqlalchemy.engines [None req-ef628b78-7123-41fa-9d73-d82e99a65691 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80108) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 07:47:55.003343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writer thread running {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 07:47:55.125753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef628b78-7123-41fa-9d73-d82e99a65691 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:47:55.130021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07f31-f684-4c70-bd8c-b05584d07ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:47:55.130689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07f31-f684-4c70-bd8c-b05584d07ead admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:55.130981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07f31-f684-4c70-bd8c-b05584d07ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:55.131258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07f31-f684-4c70-bd8c-b05584d07ead admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:55.140812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07f31-f684-4c70-bd8c-b05584d07ead admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:47:55.193786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f0d07f31-f684-4c70-bd8c-b05584d07ead admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 07:47:55.194061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07f31-f684-4c70-bd8c-b05584d07ead admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:47:55.205440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Wed Jun 29 07:47:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 07:47:55.211416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:47:55.226750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dce4bd-16d2-44d9-b394-adfa34086e1c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:47:55.231866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c0524-bfcc-459c-9429-c100429447e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:47:55.232947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c0524-bfcc-459c-9429-c100429447e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:55.233260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c0524-bfcc-459c-9429-c100429447e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:55.233541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c0524-bfcc-459c-9429-c100429447e5 admin admin] PATH_INFO: `/v3/projects/c4e7f90bb9da44679fd899e1c8b90999` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:55.237730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c0524-bfcc-459c-9429-c100429447e5 admin admin] RBAC: Authorizing `identity:get_project(project_id=c4e7f90bb9da44679fd899e1c8b90999)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:47:55.288750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5d3c0524-bfcc-459c-9429-c100429447e5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 07:47:55.289113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c0524-bfcc-459c-9429-c100429447e5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:47:55.294343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Wed Jun 29 07:47:55 2022] GET /identity/v3/projects/c4e7f90bb9da44679fd899e1c8b90999 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:47:57.331635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b711-4171-492b-8892-22886cc56a0c None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:57.332168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b711-4171-492b-8892-22886cc56a0c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:57.332266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b711-4171-492b-8892-22886cc56a0c None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:57.336352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:47:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:47:57.341962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdaab4-99cd-462b-a82e-85ffacd5c1fc None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:57.342394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdaab4-99cd-462b-a82e-85ffacd5c1fc None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:57.342521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdaab4-99cd-462b-a82e-85ffacd5c1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:57.362077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-a4bdaab4-99cd-462b-a82e-85ffacd5c1fc None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:47:57.363594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a4bdaab4-99cd-462b-a82e-85ffacd5c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:47:57.372326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:47:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 29 07:47:57.394386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34b711-4171-492b-8892-22886cc56a0c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:47:57.407337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34b711-4171-492b-8892-22886cc56a0c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:47:57.411682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d3cd3-5b08-431e-b1c3-b8f6457ad539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:47:57.412319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d3cd3-5b08-431e-b1c3-b8f6457ad539 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:57.412606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d3cd3-5b08-431e-b1c3-b8f6457ad539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:57.412885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d3cd3-5b08-431e-b1c3-b8f6457ad539 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:57.422119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d3cd3-5b08-431e-b1c3-b8f6457ad539 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:47:57.424177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d3cd3-5b08-431e-b1c3-b8f6457ad539 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:47:57.434234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Wed Jun 29 07:47:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 07:47:57.439996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdaab4-99cd-462b-a82e-85ffacd5c1fc None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:47:57.449945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdaab4-99cd-462b-a82e-85ffacd5c1fc None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:47:57.454497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7631d-ea4d-46f5-95ef-143b9432163e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:47:57.455375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7631d-ea4d-46f5-95ef-143b9432163e admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:57.455662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7631d-ea4d-46f5-95ef-143b9432163e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:57.455951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7631d-ea4d-46f5-95ef-143b9432163e admin admin] PATH_INFO: `/v3/users/b406de2279df4cff9aec118a80931d53` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:57.457427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7631d-ea4d-46f5-95ef-143b9432163e admin admin] RBAC: Authorizing `identity:get_user(user_id=b406de2279df4cff9aec118a80931d53)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:47:57.459289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7631d-ea4d-46f5-95ef-143b9432163e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:47:57.460890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Wed Jun 29 07:47:57 2022] GET /identity/v3/users/b406de2279df4cff9aec118a80931d53 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:47:59.607309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a2c9b-bf9d-4f07-8f03-99e04fa4dc45 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:59.607810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a2c9b-bf9d-4f07-8f03-99e04fa4dc45 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:59.607893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a2c9b-bf9d-4f07-8f03-99e04fa4dc45 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:59.609452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:47:59.614311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aa5c2-2c6b-4fbe-8cfd-9171f81af981 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:59.614715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aa5c2-2c6b-4fbe-8cfd-9171f81af981 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:59.614984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aa5c2-2c6b-4fbe-8cfd-9171f81af981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:59.631621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d24aa5c2-2c6b-4fbe-8cfd-9171f81af981 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:47:59.632786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d24aa5c2-2c6b-4fbe-8cfd-9171f81af981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:47:59.644592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d84fc-fe88-4667-b634-d271400c50a8 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:59.645011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d84fc-fe88-4667-b634-d271400c50a8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:59.645373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d84fc-fe88-4667-b634-d271400c50a8 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:59.646755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 6/11] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:47:59.652631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:59.652920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:59.653238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:59.694387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG passlib.handlers.bcrypt [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80108) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 07:47:59.694803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG passlib.handlers.bcrypt [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80108) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 07:47:59.718025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 6/12] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:47:59.740425 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24aa5c2-2c6b-4fbe-8cfd-9171f81af981 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:47:59.741691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:47:59.743340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:47:59.765328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:47:59.849726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 7/13] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:47:59.851248 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24aa5c2-2c6b-4fbe-8cfd-9171f81af981 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:47:59.873971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:47:59.886021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:47:59.886688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:47:59.886985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:47:59.887268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:47:59.955524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:47:59.957042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:47:59.974371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb1e45-fef4-4f38-be9a-9b1ac7123c67 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:47:59.976663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:47:59.980119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:00.013029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:00.014046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:00.014346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:00.014631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:00.028208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:00.039848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:00.040888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:00.041789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 8/14] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:00.047694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:00.068284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 7/15] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:47:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:48:00.074975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:00.150596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6326c-fda9-494c-b884-fb8f39a284a4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:00.178057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03c734-9d71-48c8-93ec-5e78f1aa2d67 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:00.188636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:00.190009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:00.190009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:00.190009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:00.190674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:00.190979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:00.192487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:00.202893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:00.205284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 9/16] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:00.210832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:00.215206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:00.218224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:00.218224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:00.218224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:00.227501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:00.229007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:00.240804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:48:00.251510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:00.252405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:00.253509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:48:00.253509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:48:00.253509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:48:00.253509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:00.253509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 8/17] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:00.258547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:00.315297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916f8e-d3f1-4170-ae30-6cf520660b5c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:00.348323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:00.349222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:00.349505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:00.349788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] PATH_INFO: `/v3/users/b406de2279df4cff9aec118a80931d53` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:00.351336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] RBAC: Authorizing `identity:get_user(user_id=b406de2279df4cff9aec118a80931d53)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:00.352797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:00.355144 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 10/18] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:00 2022] GET /identity/v3/users/b406de2279df4cff9aec118a80931d53 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:00.360247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c383e-d9eb-407c-b458-012825ce428b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:00.360493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:00.394474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:00.395138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:00.395430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:00.395702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:00.396421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:00.396724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:00.398139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:00.415450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 9/19] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:00.463020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:00.473101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642148c-7c2e-485c-86df-6438cdc1df81 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:00.523815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aa055-bf50-4c02-8ca7-8357533f2a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:00.524643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa055-bf50-4c02-8ca7-8357533f2a9e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:00.524934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa055-bf50-4c02-8ca7-8357533f2a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:00.525277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa055-bf50-4c02-8ca7-8357533f2a9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:00.527170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa055-bf50-4c02-8ca7-8357533f2a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:00.528933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa055-bf50-4c02-8ca7-8357533f2a9e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:00.530654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 11/20] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:00.536797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:00.575622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:00.648147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815915c-b3e9-45e9-a63d-5bf439bf5c3c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:00.685596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759b587-3834-4cae-a08f-f37808afc4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:00.686284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b587-3834-4cae-a08f-f37808afc4fa None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:00.686578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b587-3834-4cae-a08f-f37808afc4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:00.686862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b587-3834-4cae-a08f-f37808afc4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:00.687717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759b587-3834-4cae-a08f-f37808afc4fa None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.domain.id=default, user.id=b406de2279df4cff9aec118a80931d53) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:00.688034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759b587-3834-4cae-a08f-f37808afc4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:00.689542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759b587-3834-4cae-a08f-f37808afc4fa None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:00.697010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 10/21] 10.222.0.43 () {56 vars in 1322 bytes} [Wed Jun 29 07:48:00 2022] GET /identity/v3/role_assignments?user.id=b406de2279df4cff9aec118a80931d53&scope.domain.id=default&role.id=fe0d00714612495091430762d114121a => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:02.784864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605081-dd6c-499a-a800-b4d0506c59f2 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:02.785457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605081-dd6c-499a-a800-b4d0506c59f2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:02.785457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605081-dd6c-499a-a800-b4d0506c59f2 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:02.786914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 12/22] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:02.793565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d818-974e-46b3-83c4-2242c5083c56 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:02.794032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d818-974e-46b3-83c4-2242c5083c56 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:02.794127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d818-974e-46b3-83c4-2242c5083c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:02.822032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9b51d818-974e-46b3-83c4-2242c5083c56 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:02.823805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9b51d818-974e-46b3-83c4-2242c5083c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:02.921911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 11/23] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:02.956851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60605081-dd6c-499a-a800-b4d0506c59f2 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:03.108041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60605081-dd6c-499a-a800-b4d0506c59f2 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:03.166882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32819d3-05e7-433d-a47f-e1e33e45fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:03.167717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32819d3-05e7-433d-a47f-e1e33e45fc70 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.168006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32819d3-05e7-433d-a47f-e1e33e45fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.168286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32819d3-05e7-433d-a47f-e1e33e45fc70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.196642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32819d3-05e7-433d-a47f-e1e33e45fc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:03.198601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32819d3-05e7-433d-a47f-e1e33e45fc70 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e32819d3-05e7-433d-a47f-e1e33e45fc70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:48:03.224788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:03.225688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:03.226473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:48:03.226473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:48:03.226473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:48:03.226473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:03.226473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 13/24] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:03.234068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51d818-974e-46b3-83c4-2242c5083c56 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:03.297258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:03.357760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3b6d7-f402-4034-9f3a-d0d887ecf35d None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.358159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3b6d7-f402-4034-9f3a-d0d887ecf35d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.358398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3b6d7-f402-4034-9f3a-d0d887ecf35d None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.365113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 14/25] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:03.369801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df73bef-c998-4629-b331-1577834ddcc3 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.370145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df73bef-c998-4629-b331-1577834ddcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.370379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df73bef-c998-4629-b331-1577834ddcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.401913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3df73bef-c998-4629-b331-1577834ddcc3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:03.403154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3df73bef-c998-4629-b331-1577834ddcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:03.427720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51d818-974e-46b3-83c4-2242c5083c56 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:03.429637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:03.485342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bf148-cee0-4792-97bf-30ff2409b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:03.486052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bf148-cee0-4792-97bf-30ff2409b510 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.486348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bf148-cee0-4792-97bf-30ff2409b510 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.486630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bf148-cee0-4792-97bf-30ff2409b510 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.487385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bf148-cee0-4792-97bf-30ff2409b510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:03.487696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bf148-cee0-4792-97bf-30ff2409b510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:03.493137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bf148-cee0-4792-97bf-30ff2409b510 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:03.515067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 12/26] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:03.518377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0069-636d-4247-9288-296431e60e04 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.519818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0069-636d-4247-9288-296431e60e04 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.520245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0069-636d-4247-9288-296431e60e04 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.522584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 13/27] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:03.523668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0069-636d-4247-9288-296431e60e04 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:03.563711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 15/28] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:03.569866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4989-5e8f-44c9-b003-d57105af493f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.570400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4989-5e8f-44c9-b003-d57105af493f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.570830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4989-5e8f-44c9-b003-d57105af493f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.602164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-29df4989-5e8f-44c9-b003-d57105af493f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:03.603452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-29df4989-5e8f-44c9-b003-d57105af493f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:03.667180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:03.681883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 16/29] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:03.683505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4989-5e8f-44c9-b003-d57105af493f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:03.691182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0069-636d-4247-9288-296431e60e04 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:03.750411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37534094-5c9c-4b51-b60d-0b7b44a85467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:03.751089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37534094-5c9c-4b51-b60d-0b7b44a85467 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.751375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37534094-5c9c-4b51-b60d-0b7b44a85467 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.751650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37534094-5c9c-4b51-b60d-0b7b44a85467 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.752788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37534094-5c9c-4b51-b60d-0b7b44a85467 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:03.754615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37534094-5c9c-4b51-b60d-0b7b44a85467 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:03.790696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:03.820622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4989-5e8f-44c9-b003-d57105af493f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:03.858533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76501a4-700f-4ef2-a74e-0a1f09b2d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:03.859582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76501a4-700f-4ef2-a74e-0a1f09b2d0de None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.860047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76501a4-700f-4ef2-a74e-0a1f09b2d0de None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.860503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76501a4-700f-4ef2-a74e-0a1f09b2d0de None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.861517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76501a4-700f-4ef2-a74e-0a1f09b2d0de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:03.863099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76501a4-700f-4ef2-a74e-0a1f09b2d0de None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:03.927201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 14/30] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:48:03 2022] POST /identity/v3/projects => generated 301 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:03.929438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f83ed-f149-4ddb-9aef-6553d6d2ba77 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:03.929940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f83ed-f149-4ddb-9aef-6553d6d2ba77 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:03.930366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f83ed-f149-4ddb-9aef-6553d6d2ba77 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:03.932931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 15/31] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:03.934364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f83ed-f149-4ddb-9aef-6553d6d2ba77 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:03.999386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 17/32] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jun 29 07:48:03 2022] POST /identity/v3/roles => generated 219 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:04.000951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bc338-e8df-45ef-9bf7-aee3332405b9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.001275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bc338-e8df-45ef-9bf7-aee3332405b9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.001564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bc338-e8df-45ef-9bf7-aee3332405b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.034332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0d3bc338-e8df-45ef-9bf7-aee3332405b9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:04.036175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0d3bc338-e8df-45ef-9bf7-aee3332405b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:04.064156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f83ed-f149-4ddb-9aef-6553d6d2ba77 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:04.110154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:04.111742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.112365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.112889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.126062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:04.131259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:04.144119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:04.145072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:04.146010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:04.149539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 16/33] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:04.158095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:04.214590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 18/34] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:04.248654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bc338-e8df-45ef-9bf7-aee3332405b9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:04.312068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45390f-ca69-4924-8c9b-3795eeea0c37 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:04.356776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:04.359424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.359937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.360383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.361384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:04.361863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:04.363643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:04.367358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:04.383338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 17/35] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:04.389492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:04.432724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bc338-e8df-45ef-9bf7-aee3332405b9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:04.464089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:04.474402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d349f29-bab7-4687-9dc9-9f598de72423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:04.475024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d349f29-bab7-4687-9dc9-9f598de72423 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.475314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d349f29-bab7-4687-9dc9-9f598de72423 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.475593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d349f29-bab7-4687-9dc9-9f598de72423 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.476369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d349f29-bab7-4687-9dc9-9f598de72423 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:04.477903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d349f29-bab7-4687-9dc9-9f598de72423 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:04.501297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 19/36] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jun 29 07:48:04 2022] POST /identity/v3/roles => generated 225 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:04.504097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c4a27-2f40-4414-9840-513cacaa35c9 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.504945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c4a27-2f40-4414-9840-513cacaa35c9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.505456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c4a27-2f40-4414-9840-513cacaa35c9 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.512053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 20/37] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:04.514566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45c503-ddf3-4107-a05c-414c6d9e9df8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.514864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45c503-ddf3-4107-a05c-414c6d9e9df8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.515146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45c503-ddf3-4107-a05c-414c6d9e9df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.523778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85e4d-7a80-455c-9152-798e9b8ec3da None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:04.539546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-4e45c503-ddf3-4107-a05c-414c6d9e9df8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:04.540420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4e45c503-ddf3-4107-a05c-414c6d9e9df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:04.581950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:04.583604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.584106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.585380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] PATH_INFO: `/v3/users/b406de2279df4cff9aec118a80931d53` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.590187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] RBAC: Authorizing `identity:get_user(user_id=b406de2279df4cff9aec118a80931d53)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:04.592816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:04.595478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 18/38] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:04 2022] GET /identity/v3/users/b406de2279df4cff9aec118a80931d53 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:04.605036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:04.624919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 21/39] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:04.649843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45c503-ddf3-4107-a05c-414c6d9e9df8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:04.721734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:04.785059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45c503-ddf3-4107-a05c-414c6d9e9df8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:04.820844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf8bf9-cb96-410b-98e7-458559c1627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:04.821545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf8bf9-cb96-410b-98e7-458559c1627f None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.821840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf8bf9-cb96-410b-98e7-458559c1627f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.822117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf8bf9-cb96-410b-98e7-458559c1627f None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.822955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf8bf9-cb96-410b-98e7-458559c1627f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:04.824646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf8bf9-cb96-410b-98e7-458559c1627f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:04.837953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 22/40] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jun 29 07:48:04 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:48:04.839560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ae3c1-61cb-4a9f-a39d-0babc4c6b0b1 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.839909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ae3c1-61cb-4a9f-a39d-0babc4c6b0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.839992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75e1df-c79e-458d-9e5c-2bf44b17e6e0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:04.840199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ae3c1-61cb-4a9f-a39d-0babc4c6b0b1 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.845021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 23/41] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:04.851228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d046285-5375-491a-8abc-9e8811a9535f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.851582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d046285-5375-491a-8abc-9e8811a9535f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.851913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d046285-5375-491a-8abc-9e8811a9535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.872204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-4d046285-5375-491a-8abc-9e8811a9535f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:04.873603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4d046285-5375-491a-8abc-9e8811a9535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:04.878589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:04.885337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b392c3-393d-44dd-9484-38031290dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:04.886276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b392c3-393d-44dd-9484-38031290dade None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:04.886573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b392c3-393d-44dd-9484-38031290dade None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:04.886861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b392c3-393d-44dd-9484-38031290dade None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:04.888305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b392c3-393d-44dd-9484-38031290dade None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:04.889993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b392c3-393d-44dd-9484-38031290dade None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:04.891787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 19/42] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:04.897927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b392c3-393d-44dd-9484-38031290dade None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:04.955780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 24/43] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:04.980086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d046285-5375-491a-8abc-9e8811a9535f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:05.014318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b392c3-393d-44dd-9484-38031290dade None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:05.035498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:05.056426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:05.057429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:05.057790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:05.058078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] PATH_INFO: `/v3/domains/default/users/b406de2279df4cff9aec118a80931d53/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:05.060165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b406de2279df4cff9aec118a80931d53, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:05.061738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:05.075865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 20/44] 10.222.0.43 () {58 vars in 1361 bytes} [Wed Jun 29 07:48:04 2022] PUT /identity/v3/domains/default/users/b406de2279df4cff9aec118a80931d53/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:48:05.100270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d046285-5375-491a-8abc-9e8811a9535f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:05.140823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e399227-d80e-470c-be9b-5e929d76955d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:05.141707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399227-d80e-470c-be9b-5e929d76955d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:05.142005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399227-d80e-470c-be9b-5e929d76955d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:05.142273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399227-d80e-470c-be9b-5e929d76955d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:05.143542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e399227-d80e-470c-be9b-5e929d76955d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:05.145069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e399227-d80e-470c-be9b-5e929d76955d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:05.146749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 25/45] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:05.153277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:05.220243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:05.266203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e2fe0-e846-4e4f-9a9d-ac3accf3d9e2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:05.301292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5867142-b601-481b-b1ee-1d59786977c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:05.301969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5867142-b601-481b-b1ee-1d59786977c3 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:05.302263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5867142-b601-481b-b1ee-1d59786977c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:05.302548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5867142-b601-481b-b1ee-1d59786977c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:05.303271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5867142-b601-481b-b1ee-1d59786977c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:05.304731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5867142-b601-481b-b1ee-1d59786977c3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:05.334268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 21/46] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:48:05 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:07.177176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d57047-745a-4ac4-b85f-cb054f9820dd None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.177720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d57047-745a-4ac4-b85f-cb054f9820dd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.177720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d57047-745a-4ac4-b85f-cb054f9820dd None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.179248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 26/47] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:07.184458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2ff0-fe5e-4338-8e57-f960288ab838 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.184869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2ff0-fe5e-4338-8e57-f960288ab838 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.185057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2ff0-fe5e-4338-8e57-f960288ab838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.202857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8bfa2ff0-fe5e-4338-8e57-f960288ab838 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:07.204376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8bfa2ff0-fe5e-4338-8e57-f960288ab838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:07.273569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 22/48] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:07.298226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d57047-745a-4ac4-b85f-cb054f9820dd None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:07.401662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6b226-f5b4-4e61-b0e8-d1a2985a8783 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.401960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6b226-f5b4-4e61-b0e8-d1a2985a8783 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.402251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6b226-f5b4-4e61-b0e8-d1a2985a8783 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.403641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 23/49] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:07.408668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dbb24-1394-4a54-b0a6-31356061ca95 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.408960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dbb24-1394-4a54-b0a6-31356061ca95 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.409270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dbb24-1394-4a54-b0a6-31356061ca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.411844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d57047-745a-4ac4-b85f-cb054f9820dd None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:07.425587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:07.427466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-824dbb24-1394-4a54-b0a6-31356061ca95 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:07.428607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-824dbb24-1394-4a54-b0a6-31356061ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:07.447723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:07.449188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.449492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.449768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.460084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:07.463681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:07.468057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:07.469185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:07.469935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 27/50] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:07.475162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:07.505048 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 24/51] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:07.528499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dbb24-1394-4a54-b0a6-31356061ca95 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:07.581779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8860b-e218-4c41-b94a-b6b607a27d2a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:07.631790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dbb24-1394-4a54-b0a6-31356061ca95 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:07.668812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:07.669659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.669945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.670221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.671484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:07.673279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:07.674982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 25/52] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:07.680842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:07.720013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:07.720683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.721000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.721279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.721583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:07.722271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:07.722387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:07.724130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:07.733807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 28/53] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:07.739440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:07.773510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:07.783756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272dc85-e5bc-4582-a4d8-8f9f9b91b479 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:07.819984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:07.820644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.820940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.821257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.821981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:07.823442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:07.843514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac18c4d-07f9-4610-9f3c-f2a4e45c5baa None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:07.849532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 26/54] 10.222.0.43 () {60 vars in 1140 bytes} [Wed Jun 29 07:48:07 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:48:07.881033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:07.881948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:07.882238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:07.882517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] PATH_INFO: `/v3/users/b406de2279df4cff9aec118a80931d53` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:07.883835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] RBAC: Authorizing `identity:get_user(user_id=b406de2279df4cff9aec118a80931d53)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:07.885403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:07.887255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 29/55] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:07 2022] GET /identity/v3/users/b406de2279df4cff9aec118a80931d53 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:07.892821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:07.991711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:08.003232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77ed0f-13d5-4fe7-b109-da38b475b2bc None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:08.038595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffab918-d9d7-4ee4-bf65-2d52b1834e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:08.039411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffab918-d9d7-4ee4-bf65-2d52b1834e2a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:08.039702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffab918-d9d7-4ee4-bf65-2d52b1834e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:08.039994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffab918-d9d7-4ee4-bf65-2d52b1834e2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:08.041221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffab918-d9d7-4ee4-bf65-2d52b1834e2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:08.043353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffab918-d9d7-4ee4-bf65-2d52b1834e2a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:08.045049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 27/56] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:08.051284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:08.158008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12614c7-f066-4771-b7f0-e4d241e628cc None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:08.159703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:08.193501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90baadb1-a235-478b-ab28-ede0dd728cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:08.194155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90baadb1-a235-478b-ab28-ede0dd728cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:08.194450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90baadb1-a235-478b-ab28-ede0dd728cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:08.194732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90baadb1-a235-478b-ab28-ede0dd728cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:08.195577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90baadb1-a235-478b-ab28-ede0dd728cd0 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.domain.id=default, user.id=b406de2279df4cff9aec118a80931d53) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:08.195896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90baadb1-a235-478b-ab28-ede0dd728cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:08.197362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90baadb1-a235-478b-ab28-ede0dd728cd0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:08.206990 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 30/57] 10.222.0.43 () {56 vars in 1322 bytes} [Wed Jun 29 07:48:08 2022] GET /identity/v3/role_assignments?user.id=b406de2279df4cff9aec118a80931d53&scope.domain.id=default&role.id=fe0d00714612495091430762d114121a => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:09.915854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ee659-b9ba-47ea-a169-bcab1b3cc2d9 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:09.916908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ee659-b9ba-47ea-a169-bcab1b3cc2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:09.917058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ee659-b9ba-47ea-a169-bcab1b3cc2d9 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:09.918414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 28/58] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:09.923918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b33aca-81c4-4b6f-aa73-586e3bc31624 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:09.924312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b33aca-81c4-4b6f-aa73-586e3bc31624 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:09.924485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b33aca-81c4-4b6f-aa73-586e3bc31624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:09.941639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-57b33aca-81c4-4b6f-aa73-586e3bc31624 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:09.942835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-57b33aca-81c4-4b6f-aa73-586e3bc31624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:10.007807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 31/59] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:10.032688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ee659-b9ba-47ea-a169-bcab1b3cc2d9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:10.130906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ee659-b9ba-47ea-a169-bcab1b3cc2d9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:10.164702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c7aaf-edda-47da-956b-35266baafd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:10.165525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c7aaf-edda-47da-956b-35266baafd1e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:10.165809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c7aaf-edda-47da-956b-35266baafd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:10.166137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c7aaf-edda-47da-956b-35266baafd1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:10.167376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c7aaf-edda-47da-956b-35266baafd1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:10.168820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c7aaf-edda-47da-956b-35266baafd1e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:10.170472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 29/60] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:10.176262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b33aca-81c4-4b6f-aa73-586e3bc31624 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:10.273939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:10.280699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b33aca-81c4-4b6f-aa73-586e3bc31624 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:10.315674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addc77b-f4f8-4191-8682-51b61ce306b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:10.316361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addc77b-f4f8-4191-8682-51b61ce306b5 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:10.316657 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addc77b-f4f8-4191-8682-51b61ce306b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:10.316944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addc77b-f4f8-4191-8682-51b61ce306b5 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:10.317715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addc77b-f4f8-4191-8682-51b61ce306b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:10.319187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addc77b-f4f8-4191-8682-51b61ce306b5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:10.345578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 32/61] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:48:10 2022] POST /identity/v3/users => generated 275 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:14.144367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1e1df-ac92-49af-8351-719bd5131d2e None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.145422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1e1df-ac92-49af-8351-719bd5131d2e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.145911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1e1df-ac92-49af-8351-719bd5131d2e None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.147535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 30/62] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:14.156120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e69d-7df0-477c-aae3-c507f6b01704 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.157111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e69d-7df0-477c-aae3-c507f6b01704 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.157593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e69d-7df0-477c-aae3-c507f6b01704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.186072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-14d4e69d-7df0-477c-aae3-c507f6b01704 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:14.187712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-14d4e69d-7df0-477c-aae3-c507f6b01704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:14.259088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b07209-e165-4d24-b829-880310ca4ded None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.259743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b07209-e165-4d24-b829-880310ca4ded None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.260293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b07209-e165-4d24-b829-880310ca4ded None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.262037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 31/63] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:14.276253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f135f1-2553-4b11-b4c7-250b543681ba None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.276473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 33/64] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:14.277390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f135f1-2553-4b11-b4c7-250b543681ba None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.277874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f135f1-2553-4b11-b4c7-250b543681ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.304032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4e69d-7df0-477c-aae3-c507f6b01704 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:14.308123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c1f135f1-2553-4b11-b4c7-250b543681ba None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:14.309634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c1f135f1-2553-4b11-b4c7-250b543681ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:14.349511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:14.407358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:14.417909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 32/65] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:14.419592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c2005-f72c-4c17-9d2b-1d761d1c600b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.420081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c2005-f72c-4c17-9d2b-1d761d1c600b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.420576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c2005-f72c-4c17-9d2b-1d761d1c600b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.422199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 33/66] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:14.427949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bddc1-40c3-4e15-8a42-56e8da1a1be3 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.428505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bddc1-40c3-4e15-8a42-56e8da1a1be3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.429003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bddc1-40c3-4e15-8a42-56e8da1a1be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.450015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2d5bddc1-40c3-4e15-8a42-56e8da1a1be3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:14.451251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4e69d-7df0-477c-aae3-c507f6b01704 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:14.451836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2d5bddc1-40c3-4e15-8a42-56e8da1a1be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:14.499306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:14.500223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.500758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.501273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.514927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:14.523728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:14.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:14.525879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:14.528339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 34/67] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:14.529459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:14.543553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 34/68] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:14.544901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8d42c-5f54-4bfb-9afc-43d2ff2ca9d6 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.545224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8d42c-5f54-4bfb-9afc-43d2ff2ca9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.545504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8d42c-5f54-4bfb-9afc-43d2ff2ca9d6 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.546978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 35/69] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:14.547740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8d42c-5f54-4bfb-9afc-43d2ff2ca9d6 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:14.638192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:14.647149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f323f0-4060-431f-8f81-9d253de75da8 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:14.655094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8d42c-5f54-4bfb-9afc-43d2ff2ca9d6 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:14.685254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c96ef0-7c8a-487a-995d-e0e56a3538c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:14.686310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c96ef0-7c8a-487a-995d-e0e56a3538c6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.686776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c96ef0-7c8a-487a-995d-e0e56a3538c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.687123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c96ef0-7c8a-487a-995d-e0e56a3538c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.690916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e351d-c9f9-45b0-8932-bad0c85f1e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:14.691591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e351d-c9f9-45b0-8932-bad0c85f1e1a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.691878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e351d-c9f9-45b0-8932-bad0c85f1e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.692158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e351d-c9f9-45b0-8932-bad0c85f1e1a None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.692926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e351d-c9f9-45b0-8932-bad0c85f1e1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:14.693263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e351d-c9f9-45b0-8932-bad0c85f1e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:14.694675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e351d-c9f9-45b0-8932-bad0c85f1e1a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:14.699890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c96ef0-7c8a-487a-995d-e0e56a3538c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:14.704160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 36/70] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:14.705924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247dd0a-a991-4b36-875c-8ee837ea7e54 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.706347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247dd0a-a991-4b36-875c-8ee837ea7e54 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.706753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247dd0a-a991-4b36-875c-8ee837ea7e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-61c96ef0-7c8a-487a-995d-e0e56a3538c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:14.708433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:14.709731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 35/71] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:14.711191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab4376-56d0-482d-b2df-c93ba8a73f9f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.712594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab4376-56d0-482d-b2df-c93ba8a73f9f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.712594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab4376-56d0-482d-b2df-c93ba8a73f9f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.712836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 36/72] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:14.714450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab4376-56d0-482d-b2df-c93ba8a73f9f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:14.718111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:14.734596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5247dd0a-a991-4b36-875c-8ee837ea7e54 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:14.735843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5247dd0a-a991-4b36-875c-8ee837ea7e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:14.811404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 37/73] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:14.812286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247dd0a-a991-4b36-875c-8ee837ea7e54 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:14.825806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab4376-56d0-482d-b2df-c93ba8a73f9f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:14.874182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:14.875214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.875562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.875913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.877236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:14.879228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:14.881126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 37/74] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:14.882200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:14.906146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:14.933953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247dd0a-a991-4b36-875c-8ee837ea7e54 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:14.962921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:14.975254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a3349-933d-4ec6-9472-b19bb1e3ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:14.976149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a3349-933d-4ec6-9472-b19bb1e3ee15 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:14.976448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a3349-933d-4ec6-9472-b19bb1e3ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:14.976713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a3349-933d-4ec6-9472-b19bb1e3ee15 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:14.992581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a3349-933d-4ec6-9472-b19bb1e3ee15 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:14.995726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a3349-933d-4ec6-9472-b19bb1e3ee15 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:14.997329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538c92d-fe6a-4ae2-ae3d-87191d16866d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:14.997779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 38/75] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:14 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:14.999984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107124d-91c8-4bc7-8460-784f85dab48a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.000497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107124d-91c8-4bc7-8460-784f85dab48a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:15.001020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107124d-91c8-4bc7-8460-784f85dab48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:15.020810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9107124d-91c8-4bc7-8460-784f85dab48a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:15.022260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9107124d-91c8-4bc7-8460-784f85dab48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:15.042346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:15.043036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.043332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:15.043621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:15.044395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:15.044705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:15.046149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:15.055266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 38/76] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:15.055903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:15.099567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 39/77] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:15.100410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107124d-91c8-4bc7-8460-784f85dab48a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:15.173843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea5e0a-5113-4c00-980a-07f2937cd7f2 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:15.174598 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:15.214581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107124d-91c8-4bc7-8460-784f85dab48a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:15.229564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:15.237480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:15.238821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.239103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:15.239381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:15.251629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:15.260538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:15.262196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 39/78] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:48:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:15.263664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:15.271966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:15.273187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.273690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:15.274188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:15.275605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:15.278586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:15.323155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 40/79] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:48:15 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:15.324942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:15.416027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b62e99-cc03-468f-b588-7f86f485aceb None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:15.490631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:15.490631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.491015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:15.491562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:15.504482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6b0a1-df73-4aeb-88f4-428ce5189dd5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:15.543256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:15.567686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:15.568945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.569486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:15.569951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:15.571899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:15.574577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:15.577825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 41/80] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:15 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:15.578933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:15.594330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:15.614100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:15.614671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:15.762995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378396-be66-4eb1-9e24-b248ee8dcd72 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:15.764863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 40/81] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:15.766826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:15.825822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:15.827169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.827673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:15.828171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:15.848394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:15.865098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:15.867246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 42/82] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:15.869234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:15.901891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:15.924260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7708a0-6593-408c-8ec0-d53fd113fe5d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:15.941966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:15.997518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:15.998956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:15.999492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.000016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.001681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:16.002406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:16.004940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb524046-4e05-4f31-b674-05f72301b26a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.005258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:16.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 41/83] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:48:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:16.027500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:16.053495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:16.054463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:16.054740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.055023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.143277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:16.145107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:16.207527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:16.228346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d796-8bdf-4e60-b3a7-a210b8fa5e02 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.274407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 43/84] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:16.275305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:16.285825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:16.287391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:16.287827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.288377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.290425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:16.293743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:16.296089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 42/85] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:16 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:16.296734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:16.343444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:16.388937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abc503-f0cb-4aed-938d-eb1904e40837 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.432726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:16.433535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:16.433790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.434087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.434837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:16.435148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:16.436148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5941-8668-4730-9bea-8b8dc317208c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.436777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:16.452744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 44/86] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:16.453932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:16.473259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:16.485903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:16.487239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:16.487722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.488221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.490107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:16.491986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:16.494753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 43/87] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:16 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:16.496290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:16.584307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff5ef1-6efe-4b11-ae07-607f667b7374 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.623625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:16.624506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:16.624805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.625109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] PATH_INFO: `/v3/projects/a96947e0b4b94343b6c4ea512ceb5906` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.626406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] RBAC: Authorizing `identity:get_project(project_id=a96947e0b4b94343b6c4ea512ceb5906)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:16.627884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:16.629038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:16.631346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 45/88] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:16 2022] GET /identity/v3/projects/a96947e0b4b94343b6c4ea512ceb5906 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:16.637226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:16.654352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee967f-a32c-448b-bbfc-84f2684fae45 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.701976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:16.702673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:16.702960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.703242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.704651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=a107bd9da4da4971b4c66f789a21033b) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:16.705011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:16.706848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:16.714377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 44/89] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:16 2022] GET /identity/v3/role_assignments?user.id=a107bd9da4da4971b4c66f789a21033b&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=acffa214db594ed9803f8f5376b15d42 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:16.715609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:16.812126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:16.863507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc40172-4ec5-4dbc-8738-4a2f00751be9 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.904713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0861-2764-4b1c-96cf-ab7009e08450 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:16.910706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:16.911852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:16.912372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:16.912856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:16.950212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:17.020421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:17.023636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:17.093642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:17.094720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:17.095017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:17.095303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] PATH_INFO: `/v3/users/b406de2279df4cff9aec118a80931d53` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:17.096876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] RBAC: Authorizing `identity:get_user(user_id=b406de2279df4cff9aec118a80931d53)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:17.098503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:17.100707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 46/90] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:16 2022] GET /identity/v3/users/b406de2279df4cff9aec118a80931d53 => generated 247 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:17.103612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:17.159001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 45/91] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:17.160252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:17.279946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:17.343613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3865d-815e-4dd3-bdb2-802e29104e97 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:17.401626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:17.402315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:17.402606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:17.402886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:17.403751 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=a96947e0b4b94343b6c4ea512ceb5906, user.id=a107bd9da4da4971b4c66f789a21033b) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:17.404057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:17.405570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:17.406648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47915f1-aa7b-45f3-98c4-4c93689e394f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:17.412156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 47/92] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:17 2022] GET /identity/v3/role_assignments?user.id=a107bd9da4da4971b4c66f789a21033b&scope.project.id=a96947e0b4b94343b6c4ea512ceb5906&role.id=acffa214db594ed9803f8f5376b15d42 => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:17.412725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:17.449471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:17.565931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:17.566923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:17.567043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:17.567276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:17.582916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c7102-5207-4c57-bd93-49ebbf3b40d3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:17.647176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:17.648343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:17.648771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:17.649224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:17.650793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:17.652603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:17.654601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 48/93] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:17 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:17.666855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:17.715291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:17.716895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:17.816154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:17.876314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:17.909442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 46/94] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:17.919514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:17.928075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d76f49-1a4f-4734-b56d-babb5c544f2a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:18.050550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:18.051264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.051564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.051852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.052753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] RBAC: Adding query filter params (role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=a107bd9da4da4971b4c66f789a21033b) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:18.053107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:18.054716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:18.079016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 49/95] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:17 2022] GET /identity/v3/role_assignments?user.id=a107bd9da4da4971b4c66f789a21033b&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 259 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:18.105150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d35956-59ea-44b6-853e-cb9f2a1154f0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:18.166374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cab6b-ec38-4b49-a1a7-04cab040db43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:18.167367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cab6b-ec38-4b49-a1a7-04cab040db43 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.167658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cab6b-ec38-4b49-a1a7-04cab040db43 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.168125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cab6b-ec38-4b49-a1a7-04cab040db43 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.174181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cab6b-ec38-4b49-a1a7-04cab040db43 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:18.175613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cab6b-ec38-4b49-a1a7-04cab040db43 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:18.177764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 47/96] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:17 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:18.184812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:18.309849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b7105-e548-47b1-9df1-c3bceb2e381f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.310363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b7105-e548-47b1-9df1-c3bceb2e381f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.310434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b7105-e548-47b1-9df1-c3bceb2e381f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.312003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 48/97] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:18.318806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaeac2-1e41-4a5f-883e-893c9a037e46 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.319095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaeac2-1e41-4a5f-883e-893c9a037e46 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.319382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaeac2-1e41-4a5f-883e-893c9a037e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.354968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3dbaeac2-1e41-4a5f-883e-893c9a037e46 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:18.356198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3dbaeac2-1e41-4a5f-883e-893c9a037e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:18.403425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:18.414430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a09e0-6552-4342-b9f1-09237b3ebcc3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:18.478983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aade38-2ba2-4232-99c3-00bc85ef83f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:18.479575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aade38-2ba2-4232-99c3-00bc85ef83f6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.479869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aade38-2ba2-4232-99c3-00bc85ef83f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.480147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aade38-2ba2-4232-99c3-00bc85ef83f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.489294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aade38-2ba2-4232-99c3-00bc85ef83f6 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=b406de2279df4cff9aec118a80931d53) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:18.489294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aade38-2ba2-4232-99c3-00bc85ef83f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:18.489294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aade38-2ba2-4232-99c3-00bc85ef83f6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:18.497089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 50/98] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:18 2022] GET /identity/v3/role_assignments?user.id=b406de2279df4cff9aec118a80931d53&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:18.564764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 49/99] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:18.576866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b144c-6f7c-4933-a7fe-45433b1d2b40 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.577203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b144c-6f7c-4933-a7fe-45433b1d2b40 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.577493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b144c-6f7c-4933-a7fe-45433b1d2b40 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.579273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 51/100] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:18.585125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89e75f-d64c-4d9d-a41c-9f78608870c4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.585427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89e75f-d64c-4d9d-a41c-9f78608870c4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.585889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89e75f-d64c-4d9d-a41c-9f78608870c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.608843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3d89e75f-d64c-4d9d-a41c-9f78608870c4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:18.610099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3d89e75f-d64c-4d9d-a41c-9f78608870c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:18.615930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b144c-6f7c-4933-a7fe-45433b1d2b40 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:18.633232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:18.739470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 50/101] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:18.774834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b144c-6f7c-4933-a7fe-45433b1d2b40 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:18.781934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89e75f-d64c-4d9d-a41c-9f78608870c4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:18.835732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81115276-707a-4452-926a-de5cc8084361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:18.836573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81115276-707a-4452-926a-de5cc8084361 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:18.836870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81115276-707a-4452-926a-de5cc8084361 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:18.837167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81115276-707a-4452-926a-de5cc8084361 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:18.841791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81115276-707a-4452-926a-de5cc8084361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:18.843315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81115276-707a-4452-926a-de5cc8084361 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:18.845450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 52/102] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:18.854764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81115276-707a-4452-926a-de5cc8084361 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:18.970811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:18.993436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:19.003315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89e75f-d64c-4d9d-a41c-9f78608870c4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:19.063526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:19.064333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:19.064624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:19.064905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:19.082051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81115276-707a-4452-926a-de5cc8084361 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:19.086494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:19.101203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:19.102101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:19.102840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 51/103] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:19.134191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:19.148570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:19.149306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:19.149598 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:19.149881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:19.150738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:19.152228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:19.195609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 53/104] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:48:18 2022] POST /identity/v3/projects => generated 302 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:48:19.312266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2e894-d82c-4f52-9cf3-869e0bd2cd03 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:19.360646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:19.361425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:19.361709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:19.361993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:19.362775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:19.363088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:19.364556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:19.378356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 52/105] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:19.384074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:19.579364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fbf41-b253-4108-81e9-cdce8dc189e3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:19.629701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:19.639909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:19.640873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:19.641190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:19.641470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] PATH_INFO: `/v3/users/c4e1a26d9c194e1f9b065b13134fc530` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:19.663500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] RBAC: Authorizing `identity:get_user(user_id=c4e1a26d9c194e1f9b065b13134fc530)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:19.665019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:19.667031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 54/106] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:19 2022] GET /identity/v3/users/c4e1a26d9c194e1f9b065b13134fc530 => generated 289 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:19.673127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:19.677415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:19.822168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fba42-0b73-4226-b4e1-c10724519a07 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:19.870918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06855e4-a426-4b3d-9068-eae81c3c5909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:19.871512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06855e4-a426-4b3d-9068-eae81c3c5909 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:19.871798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06855e4-a426-4b3d-9068-eae81c3c5909 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:19.872076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06855e4-a426-4b3d-9068-eae81c3c5909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:19.968752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06855e4-a426-4b3d-9068-eae81c3c5909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:19.970485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06855e4-a426-4b3d-9068-eae81c3c5909 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:19.974142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:20.108663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 53/107] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:20.115004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:20.291612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075361e2-2a9f-406d-97d7-2318a5b35f7e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:20.296937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b38b2-f32e-4542-8f32-f650ecd9daec None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:20.297298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b38b2-f32e-4542-8f32-f650ecd9daec None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:20.297799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b38b2-f32e-4542-8f32-f650ecd9daec None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:20.299452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 54/108] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:20.311934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de093-242b-4b3a-950f-64c5ea4543ba None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:20.312451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de093-242b-4b3a-950f-64c5ea4543ba None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:20.313188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de093-242b-4b3a-950f-64c5ea4543ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:20.358135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:20.359038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:20.359322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:20.359600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:20.361871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:20.362143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-113de093-242b-4b3a-950f-64c5ea4543ba None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:20.363657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:20.364240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-113de093-242b-4b3a-950f-64c5ea4543ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:20.365685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 55/109] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:20 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:20.373750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:20.513361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:20.560549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 55/110] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:20.587008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a86d5f-4bd9-441d-a5dc-5822c1057f56 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:20.602517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de093-242b-4b3a-950f-64c5ea4543ba None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:20.627258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17665fd7-f87f-4cd4-9a9e-29bf8304eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:20.628298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17665fd7-f87f-4cd4-9a9e-29bf8304eca4 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:20.628792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17665fd7-f87f-4cd4-9a9e-29bf8304eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:20.629338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17665fd7-f87f-4cd4-9a9e-29bf8304eca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:20.630580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17665fd7-f87f-4cd4-9a9e-29bf8304eca4 None admin] RBAC: Adding query filter params (role.id=00ac4a79282e459f8e0aa6c551f0322d, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=c4e1a26d9c194e1f9b065b13134fc530) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:20.631135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17665fd7-f87f-4cd4-9a9e-29bf8304eca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:20.633009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17665fd7-f87f-4cd4-9a9e-29bf8304eca4 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:20.640455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 56/111] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:20 2022] GET /identity/v3/role_assignments?user.id=c4e1a26d9c194e1f9b065b13134fc530&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=00ac4a79282e459f8e0aa6c551f0322d => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:20.688899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:20.740353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e998-2bca-481b-9c4b-614a26ac6383 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:20.741151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e998-2bca-481b-9c4b-614a26ac6383 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:20.741656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e998-2bca-481b-9c4b-614a26ac6383 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:20.743804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 57/112] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:20.752062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffada2-a9b2-44f7-b29f-860e310389e9 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:20.752709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffada2-a9b2-44f7-b29f-860e310389e9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:20.753384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffada2-a9b2-44f7-b29f-860e310389e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:20.784594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-1dffada2-a9b2-44f7-b29f-860e310389e9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:20.786263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1dffada2-a9b2-44f7-b29f-860e310389e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:20.809963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de093-242b-4b3a-950f-64c5ea4543ba None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:20.868385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:20.870132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:20.870643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:20.871147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:20.888621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:20.900932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:20.902654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:20.903889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:20.909860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 56/113] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:20.918859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:20.950377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 58/114] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:20.988097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffada2-a9b2-44f7-b29f-860e310389e9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:21.097804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:21.137683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:21.155142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b709c3-cdff-4f30-8c27-bc4e6c626aa4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.165581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffada2-a9b2-44f7-b29f-860e310389e9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.206029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:21.206856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.207145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.207430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.219920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:21.233726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:21.234422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.234708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:21.234940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:21.235750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:21.236499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:21.237090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.237090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:21.237090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:21.237279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 59/115] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:21.237844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:21.242586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:21.255451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 57/116] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:21.262099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:21.362354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d721ab-0d04-4ce1-acf6-731a27eb1e46 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.385504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:21.445312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:21.447310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.447707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.448216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.450331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:21.450658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:21.452145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:21.457139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3c31f-2559-43bb-b7a3-77c9c72c02ca None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.474811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 60/117] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:21.497018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:21.552552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:21.554621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.554914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.555198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.559122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:21.561560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:21.563639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 58/118] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:21 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:21.569392 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:21.628262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:21.676099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2badf-92fd-486f-bbc9-bbdbeae3b66a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.717803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d51161-1e53-432b-9243-d2657b8beb06 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.718173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa368bb-35ca-4112-a9f8-90de0b3feb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:21.719366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa368bb-35ca-4112-a9f8-90de0b3feb00 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.719875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa368bb-35ca-4112-a9f8-90de0b3feb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.720355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa368bb-35ca-4112-a9f8-90de0b3feb00 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.722138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa368bb-35ca-4112-a9f8-90de0b3feb00 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:21.723837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa368bb-35ca-4112-a9f8-90de0b3feb00 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:21.725922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 61/119] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:21 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:21.727470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b5890-e55b-4c7d-95be-15759a8fd3ca None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.727962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b5890-e55b-4c7d-95be-15759a8fd3ca None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.728529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b5890-e55b-4c7d-95be-15759a8fd3ca None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.730125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 62/120] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:21.732970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b5890-e55b-4c7d-95be-15759a8fd3ca None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:21.763009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bde77-c247-4550-9db8-56f04177b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:21.763898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bde77-c247-4550-9db8-56f04177b89f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.764212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bde77-c247-4550-9db8-56f04177b89f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.764460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bde77-c247-4550-9db8-56f04177b89f None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.765806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bde77-c247-4550-9db8-56f04177b89f None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:21.767180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bde77-c247-4550-9db8-56f04177b89f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:21.769420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 59/121] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:21 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:21.770634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9817-5684-402f-98f9-cb96afb9d0c5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.770924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9817-5684-402f-98f9-cb96afb9d0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.771207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9817-5684-402f-98f9-cb96afb9d0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.801270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:21.801454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-749e9817-5684-402f-98f9-cb96afb9d0c5 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:21.801454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-749e9817-5684-402f-98f9-cb96afb9d0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:21.873618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 60/122] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:21.874470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e9817-5684-402f-98f9-cb96afb9d0c5 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:21.899753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:21.910238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b5890-e55b-4c7d-95be-15759a8fd3ca None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.976847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:21.978192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:21.979352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:21.979848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] PATH_INFO: `/v3/projects/a96947e0b4b94343b6c4ea512ceb5906` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:21.981861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] RBAC: Authorizing `identity:get_project(project_id=a96947e0b4b94343b6c4ea512ceb5906)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:21.983754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:21.985345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e9817-5684-402f-98f9-cb96afb9d0c5 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:21.986083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 63/123] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:21 2022] GET /identity/v3/projects/a96947e0b4b94343b6c4ea512ceb5906 => generated 312 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:21.992546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:22.033198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:22.033198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.033198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.033198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/a107bd9da4da4971b4c66f789a21033b/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.033607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] RBAC: Authorizing `identity:create_grant(project_id=6bf43a1fb0074f3c8f0a85646b793ebc, user_id=a107bd9da4da4971b4c66f789a21033b, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:22.033607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:22.054515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 61/124] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:21 2022] PUT /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/a107bd9da4da4971b4c66f789a21033b/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:22.055476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:22.228053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744bea5-d583-414d-bc0f-99420f30b415 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:22.237726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:22.242361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:22.267745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354f3fa-8e6b-4dce-a86a-054049204b4d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:22.275651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f28f6-30f9-4ba2-b3b2-5ba5e6ae26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:22.276488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f28f6-30f9-4ba2-b3b2-5ba5e6ae26a8 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.276777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f28f6-30f9-4ba2-b3b2-5ba5e6ae26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.277088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f28f6-30f9-4ba2-b3b2-5ba5e6ae26a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.288267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f28f6-30f9-4ba2-b3b2-5ba5e6ae26a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-4d9f28f6-30f9-4ba2-b3b2-5ba5e6ae26a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:22.297106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:22.298244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:22.298994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 64/125] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:48:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:22.299835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09da5cb-17e7-4fef-9177-df9693846dff None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.300126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09da5cb-17e7-4fef-9177-df9693846dff None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.300404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09da5cb-17e7-4fef-9177-df9693846dff None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.301913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 65/126] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:22.305813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09da5cb-17e7-4fef-9177-df9693846dff None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:22.314413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ae326-b47d-4b12-94f2-c74424353193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:22.315414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ae326-b47d-4b12-94f2-c74424353193 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.315776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ae326-b47d-4b12-94f2-c74424353193 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.316131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ae326-b47d-4b12-94f2-c74424353193 None admin] PATH_INFO: `/v3/projects/a96947e0b4b94343b6c4ea512ceb5906/users/a107bd9da4da4971b4c66f789a21033b/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.318352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ae326-b47d-4b12-94f2-c74424353193 None admin] RBAC: Authorizing `identity:create_grant(project_id=a96947e0b4b94343b6c4ea512ceb5906, user_id=a107bd9da4da4971b4c66f789a21033b, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:22.319950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ae326-b47d-4b12-94f2-c74424353193 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:22.336233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 62/127] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:22 2022] PUT /identity/v3/projects/a96947e0b4b94343b6c4ea512ceb5906/users/a107bd9da4da4971b4c66f789a21033b/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:22.340052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801976-2adf-4d82-b898-49828a06c7b1 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.340352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801976-2adf-4d82-b898-49828a06c7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.340723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801976-2adf-4d82-b898-49828a06c7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.371688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b7801976-2adf-4d82-b898-49828a06c7b1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:22.373136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b7801976-2adf-4d82-b898-49828a06c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:22.424397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09da5cb-17e7-4fef-9177-df9693846dff None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:22.469206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:22.469954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.470243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.470527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.472385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:22.473707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:22.477316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:22.492594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 66/128] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:48:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:22.500605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:22.502014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 63/129] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:22.589595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7801976-2adf-4d82-b898-49828a06c7b1 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:22.592338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:22.687110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da373e-a234-4eac-8df7-69c761e61a23 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:22.701448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:22.740196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7801976-2adf-4d82-b898-49828a06c7b1 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:22.751702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:22.752771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.753082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.753365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.759212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:22.760877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:22.769434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 67/130] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:22 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:22.781138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:22.829815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:22.830626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:22.830914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:22.831195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:22.858308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:22.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:22.870006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:22.870762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:22.872119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 64/131] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:22.877321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:22.999410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571a34-0f81-4246-b4c7-7f334aabc9f6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:23.013379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:23.036082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3b8cc-a416-4064-8240-5eeb4c02f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:23.037016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3b8cc-a416-4064-8240-5eeb4c02f426 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.037394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3b8cc-a416-4064-8240-5eeb4c02f426 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.037689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3b8cc-a416-4064-8240-5eeb4c02f426 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.038921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3b8cc-a416-4064-8240-5eeb4c02f426 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:23.040472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3b8cc-a416-4064-8240-5eeb4c02f426 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:23.043033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 68/132] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:22 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:23.044361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6746b-c671-46a8-a4a8-dcd328b3be26 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.044658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6746b-c671-46a8-a4a8-dcd328b3be26 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.044946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6746b-c671-46a8-a4a8-dcd328b3be26 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.046372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 69/133] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:23.051643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6746b-c671-46a8-a4a8-dcd328b3be26 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:23.162066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6746b-c671-46a8-a4a8-dcd328b3be26 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:23.162976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb4d1b-7113-4160-827b-6fe80832d97c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:23.199008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce148a52-e009-43d2-980c-c76751976d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:23.199912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148a52-e009-43d2-980c-c76751976d86 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.200193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148a52-e009-43d2-980c-c76751976d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.200467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148a52-e009-43d2-980c-c76751976d86 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/a107bd9da4da4971b4c66f789a21033b/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.208057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce148a52-e009-43d2-980c-c76751976d86 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bf43a1fb0074f3c8f0a85646b793ebc, user_id=a107bd9da4da4971b4c66f789a21033b, role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:23.210283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce148a52-e009-43d2-980c-c76751976d86 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:23.229396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:23.236960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 70/134] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:23 2022] PUT /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/a107bd9da4da4971b4c66f789a21033b/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:23.238401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09b37-6828-42d1-a733-2887a428908c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.239798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09b37-6828-42d1-a733-2887a428908c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.240088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09b37-6828-42d1-a733-2887a428908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.246198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:23.247681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.247973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.248257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.251400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:23.251705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:23.254209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:23.274603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 65/135] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:23.274894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b7a09b37-6828-42d1-a733-2887a428908c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:23.276971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b7a09b37-6828-42d1-a733-2887a428908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:23.282584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:23.323032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:23.427575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 71/136] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:23.471187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a09b37-6828-42d1-a733-2887a428908c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:23.519919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8081471-0bfb-41f2-bee5-542564d4eb34 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:23.582000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:23.584047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.586312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.587102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] PATH_INFO: `/v3/users/b406de2279df4cff9aec118a80931d53` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.592958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] RBAC: Authorizing `identity:get_user(user_id=b406de2279df4cff9aec118a80931d53)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:23.595783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:23.604969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 66/137] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:23 2022] GET /identity/v3/users/b406de2279df4cff9aec118a80931d53 => generated 247 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:23.611506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:23.774109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a09b37-6828-42d1-a733-2887a428908c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:23.797347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:23.875980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:23.876860 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.877174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.877456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.879396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:23.880870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:23.889530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 72/138] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:23.892816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3f2a-73e8-46e9-9194-921739da3c9a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:23.893851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:23.945145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:23.968938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075c6ed-dddb-4c9a-b9bb-cb81a80f3963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:23.970650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075c6ed-dddb-4c9a-b9bb-cb81a80f3963 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.971182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075c6ed-dddb-4c9a-b9bb-cb81a80f3963 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.971693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075c6ed-dddb-4c9a-b9bb-cb81a80f3963 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.973535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075c6ed-dddb-4c9a-b9bb-cb81a80f3963 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:23.976422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075c6ed-dddb-4c9a-b9bb-cb81a80f3963 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:23.980752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 67/139] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:23 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:23.983143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c15c-9b84-4632-a1b8-358f378b82ca None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:23.983543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c15c-9b84-4632-a1b8-358f378b82ca None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:23.984055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c15c-9b84-4632-a1b8-358f378b82ca None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:23.987207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 68/140] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:23.988698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c15c-9b84-4632-a1b8-358f378b82ca None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:24.079969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b85f5e-6f7d-4bd7-96da-b00d8839b528 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:24.141707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f3493-233a-4276-add8-2a81dbffab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:24.142773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f3493-233a-4276-add8-2a81dbffab9e None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:24.143276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f3493-233a-4276-add8-2a81dbffab9e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:24.143781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f3493-233a-4276-add8-2a81dbffab9e None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:24.144722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f3493-233a-4276-add8-2a81dbffab9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:24.146431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f3493-233a-4276-add8-2a81dbffab9e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:24.167420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c15c-9b84-4632-a1b8-358f378b82ca None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:24.184565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 73/141] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:48:23 2022] POST /identity/v3/users => generated 283 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:48:24.186374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be296e-9427-4fe0-b237-02d442b3b71b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:24.186896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be296e-9427-4fe0-b237-02d442b3b71b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:24.187377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be296e-9427-4fe0-b237-02d442b3b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:24.213321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:24.225462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:24.226149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-01be296e-9427-4fe0-b237-02d442b3b71b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:24.226468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:24.226762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:24.227046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/b406de2279df4cff9aec118a80931d53/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:24.227421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-01be296e-9427-4fe0-b237-02d442b3b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:24.229950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bf43a1fb0074f3c8f0a85646b793ebc, user_id=b406de2279df4cff9aec118a80931d53, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:24.231896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:24.267119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 69/142] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:23 2022] PUT /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/b406de2279df4cff9aec118a80931d53/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:24.361828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 74/143] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:24.400019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:24.609693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0e30-91a9-4225-b4b2-d6590f67eab2 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:24.672042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:24.673621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:24.674135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:24.674642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:24.695035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:24.711136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:24.712050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:24.713354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:24.714383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 70/144] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:24.721321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be296e-9427-4fe0-b237-02d442b3b71b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:24.733690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:24.860485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be296e-9427-4fe0-b237-02d442b3b71b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:24.905518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c5773-1041-4ad6-a1fd-c899e19ff253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:24.906207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5773-1041-4ad6-a1fd-c899e19ff253 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:24.906487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5773-1041-4ad6-a1fd-c899e19ff253 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:24.906825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5773-1041-4ad6-a1fd-c899e19ff253 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:24.907529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c5773-1041-4ad6-a1fd-c899e19ff253 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:24.907831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c5773-1041-4ad6-a1fd-c899e19ff253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:24.909262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c5773-1041-4ad6-a1fd-c899e19ff253 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:24.921473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 75/145] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:24.927928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:24.958919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7955bf8-09db-47a5-9ec6-fe699d621a88 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:24.959590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7955bf8-09db-47a5-9ec6-fe699d621a88 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:24.960087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7955bf8-09db-47a5-9ec6-fe699d621a88 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:24.962003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 76/146] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:24.968023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216cf4-929c-439c-a25d-8d7e576fb829 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:24.968691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216cf4-929c-439c-a25d-8d7e576fb829 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:24.969242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216cf4-929c-439c-a25d-8d7e576fb829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:24.991526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-94216cf4-929c-439c-a25d-8d7e576fb829 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:24.993252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-94216cf4-929c-439c-a25d-8d7e576fb829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:25.065800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:25.116057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:25.126244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 77/147] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:25.156681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94216cf4-929c-439c-a25d-8d7e576fb829 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:25.221311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b0e79-e966-4ce0-8142-fbfff93b56aa None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:25.324667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:25.326254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:25.326719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:25.327344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] PATH_INFO: `/v3/users/c4e1a26d9c194e1f9b065b13134fc530` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:25.329665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] RBAC: Authorizing `identity:get_user(user_id=c4e1a26d9c194e1f9b065b13134fc530)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:25.332301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:25.335842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 71/148] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:24 2022] GET /identity/v3/users/c4e1a26d9c194e1f9b065b13134fc530 => generated 289 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:25.342769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:25.424565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94216cf4-929c-439c-a25d-8d7e576fb829 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:25.545848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:25.547327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:25.548087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:25.548595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:25.590205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:25.598838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:25.599749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:25.600490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:25.607257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 78/149] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:25.613382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:25.693327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149ff2-567c-4e11-a5d4-4ef5db52cac4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:25.725025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:25.735195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:25.754326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf911b-ad89-45ed-b30d-e0d0f62500e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:25.755467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf911b-ad89-45ed-b30d-e0d0f62500e5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:25.755928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf911b-ad89-45ed-b30d-e0d0f62500e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:25.756389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf911b-ad89-45ed-b30d-e0d0f62500e5 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:25.761998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf911b-ad89-45ed-b30d-e0d0f62500e5 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:25.763740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf911b-ad89-45ed-b30d-e0d0f62500e5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:25.766222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 72/150] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:25 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:25.767885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370cf5c-09f3-4c6d-b781-1b94b5c0570a None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:25.768458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370cf5c-09f3-4c6d-b781-1b94b5c0570a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:25.768916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370cf5c-09f3-4c6d-b781-1b94b5c0570a None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:25.770533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 73/151] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:25.778587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85ad4-9be7-4a9d-8b6f-9d98119ff89c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:25.779149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85ad4-9be7-4a9d-8b6f-9d98119ff89c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:25.779597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85ad4-9be7-4a9d-8b6f-9d98119ff89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:25.795668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7b014-b38e-44fc-86ad-a2daca814eb3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:25.811709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8ab85ad4-9be7-4a9d-8b6f-9d98119ff89c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:25.813227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8ab85ad4-9be7-4a9d-8b6f-9d98119ff89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:25.834961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:25.835914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:25.836404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:25.836759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:25.837699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:25.838139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:25.840127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:25.851254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 79/152] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:25.852297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:25.936817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 74/153] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:25.938084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85ad4-9be7-4a9d-8b6f-9d98119ff89c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:25.989893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccead2-1121-4ceb-b87c-97140a882315 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:26.021536 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:26.037570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:26.038866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:26.039229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:26.039587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/c4e1a26d9c194e1f9b065b13134fc530/roles/00ac4a79282e459f8e0aa6c551f0322d` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:26.043582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bf43a1fb0074f3c8f0a85646b793ebc, user_id=c4e1a26d9c194e1f9b065b13134fc530, role_id=00ac4a79282e459f8e0aa6c551f0322d)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:26.045712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:26.073852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 80/154] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:25 2022] PUT /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/users/c4e1a26d9c194e1f9b065b13134fc530/roles/00ac4a79282e459f8e0aa6c551f0322d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:26.074844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:26.118035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85ad4-9be7-4a9d-8b6f-9d98119ff89c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:26.146023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:26.186528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:26.188090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:26.188558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:26.189031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:26.194555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:26.196273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:26.202580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 75/155] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:25 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:26.211398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:26.332889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c942a15-cea3-42fa-a0eb-e3c8736099a6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:26.434722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d3a62-9f9e-4b86-a5a5-3dbbd956fd9d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:26.508296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:26.510748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:26.511244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:26.511708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:26.531253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:26.532307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:26.532786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:26.533282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:26.554602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:26.573971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:26.575045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:26.575897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:26.576480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 81/156] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 502 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:26.583679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:26.637366 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:26.650771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:26.652273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:26.737508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:26.763342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 76/157] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:26.769211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:26.873621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:26.889326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4d85d-3eb8-4671-a711-798f5c23dc73 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:26.900994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd98fb0-ee37-483a-93dc-1e167e7b4453 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:26.953785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:26.955052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:26.955531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:26.955995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:26.961737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:26.963982 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:26.966029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 77/158] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:26 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:26.974263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:26.998194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:26.999341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:26.999820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:27.000316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:27.001743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:27.002278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:27.004776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:27.020314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 82/159] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:27.027464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:27.142025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d0f18-5035-4b8c-b35b-63966e2e753b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:27.199548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:27.200231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:27.200515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:27.200788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:27.201669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=a107bd9da4da4971b4c66f789a21033b) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:27.201985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:27.203436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:27.214344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 78/160] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:26 2022] GET /identity/v3/role_assignments?user.id=a107bd9da4da4971b4c66f789a21033b&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=acffa214db594ed9803f8f5376b15d42 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:27.305539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b29563-51f0-4a1e-b376-f6d152571276 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:27.319848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:27.371304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4ac5b-eefe-4b20-af62-543f9902ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:27.372891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4ac5b-eefe-4b20-af62-543f9902ab87 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:27.373435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4ac5b-eefe-4b20-af62-543f9902ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:27.373922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4ac5b-eefe-4b20-af62-543f9902ab87 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:27.376269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4ac5b-eefe-4b20-af62-543f9902ab87 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:27.379071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4ac5b-eefe-4b20-af62-543f9902ab87 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:27.382612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 83/161] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:27 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:27.389381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:27.427585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:27.533923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8349e-badf-4e19-b5b4-df55d8e1fb8c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:27.577197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:27.577865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:27.578148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:27.578424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:27.625345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af54e-8320-4590-bdc0-f82c624c85e3 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:27.625997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af54e-8320-4590-bdc0-f82c624c85e3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:27.633007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af54e-8320-4590-bdc0-f82c624c85e3 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:27.635672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 84/162] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:27 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:27.641948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0da4-b565-4c52-af0b-568d03b87982 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:27.642587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0da4-b565-4c52-af0b-568d03b87982 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:27.643065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0da4-b565-4c52-af0b-568d03b87982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:27.674262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:27.685109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:27.699098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7c5c0da4-b565-4c52-af0b-568d03b87982 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:27.701086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7c5c0da4-b565-4c52-af0b-568d03b87982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:27.733537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:27.830735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 85/163] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:27.871268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0da4-b565-4c52-af0b-568d03b87982 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:27.885790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 79/164] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:48:27.901619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:28.180816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0da4-b565-4c52-af0b-568d03b87982 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:28.185857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:28.217349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525cec2-45f4-494f-9d42-cc75a7518727 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:28.261520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:28.287623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:28.288282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:28.288552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:28.288784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:28.289085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] PATH_INFO: `/v3/projects/a96947e0b4b94343b6c4ea512ceb5906` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:28.289804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:28.290310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:28.291451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:28.291603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] RBAC: Authorizing `identity:get_project(project_id=a96947e0b4b94343b6c4ea512ceb5906)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:28.293231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:28.295287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 80/165] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:27 2022] GET /identity/v3/projects/a96947e0b4b94343b6c4ea512ceb5906 => generated 312 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:28.304535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:28.330361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:28.359330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:28.361120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:28.363531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 86/166] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:48:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 491 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:28.368107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:28.476925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a348ce-baea-4ccb-a7b3-2449e513c811 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:28.548737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:28.549908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:28.550393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:28.550880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:28.552442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=a96947e0b4b94343b6c4ea512ceb5906, user.id=a107bd9da4da4971b4c66f789a21033b) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:28.553034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:28.555660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:28.576233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 81/167] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:28 2022] GET /identity/v3/role_assignments?user.id=a107bd9da4da4971b4c66f789a21033b&scope.project.id=a96947e0b4b94343b6c4ea512ceb5906&role.id=acffa214db594ed9803f8f5376b15d42 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:28.662977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a9cf-5ac0-4539-b6f5-0228f289c7f5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:28.723249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196ae93-1f2c-4cd7-9314-e4d643216ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:28.724387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ae93-1f2c-4cd7-9314-e4d643216ffd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:28.724872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ae93-1f2c-4cd7-9314-e4d643216ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:28.725371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ae93-1f2c-4cd7-9314-e4d643216ffd None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:28.726383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196ae93-1f2c-4cd7-9314-e4d643216ffd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:28.726907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196ae93-1f2c-4cd7-9314-e4d643216ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:28.729841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196ae93-1f2c-4cd7-9314-e4d643216ffd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:28.746930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 87/168] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:48:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:28.755820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:28.864194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:28.973324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4307c-5f7b-4b6a-b3e4-94d75d390b2b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:29.054473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:29.056002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:29.056566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:29.057031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] PATH_INFO: `/v3/users/a107bd9da4da4971b4c66f789a21033b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:29.062277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] RBAC: Authorizing `identity:get_user(user_id=a107bd9da4da4971b4c66f789a21033b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:29.065132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:29.068299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 82/169] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:28 2022] GET /identity/v3/users/a107bd9da4da4971b4c66f789a21033b => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:29.070760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b379ff9-f264-4e08-9306-3f41b9fa0673 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:29.071272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b379ff9-f264-4e08-9306-3f41b9fa0673 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:29.071767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b379ff9-f264-4e08-9306-3f41b9fa0673 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:29.074555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 88/170] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:29.077974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:29.121006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb79ef-2661-4455-94b2-b60aba0920a2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:29.121634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb79ef-2661-4455-94b2-b60aba0920a2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:29.122110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb79ef-2661-4455-94b2-b60aba0920a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:29.153302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-1edb79ef-2661-4455-94b2-b60aba0920a2 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:29.154881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1edb79ef-2661-4455-94b2-b60aba0920a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:29.196193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:29.303491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3389254-b536-457a-9912-7551b19a9b14 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:29.326913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 89/171] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:29.342253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:29.369535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:29.370588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:29.371570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:29.372122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:29.401099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb79ef-2661-4455-94b2-b60aba0920a2 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:29.583214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:29.585511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:29.693348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb79ef-2661-4455-94b2-b60aba0920a2 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:29.785113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:29.785942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:29.786284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:29.786560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:29.798793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 83/172] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:29.804604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:29.816353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:29.824885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:29.825824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:29.826612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:29.834442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 90/173] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:29.840901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:29.856583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:29.872210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:30.018663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdbecc-4ed7-4ad3-8399-be0da59a2ddd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:30.056867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:30.057846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.058320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.058835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.059760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:30.060216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:30.061805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:30.071280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 91/174] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:30.082045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:30.096462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb020aed-0cb7-4db1-b1bc-0719cce5614e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:30.174668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:30.175800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.176230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.176678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.178867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:30.183033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:30.187684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 84/175] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:29 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:30.207081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:30.229732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:30.255373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d25e75-bddb-4178-9a3a-ddd7e547e4c7 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:30.316996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:30.318301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.318815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.320270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] PATH_INFO: `/v3/users/b406de2279df4cff9aec118a80931d53` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.323550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] RBAC: Authorizing `identity:get_user(user_id=b406de2279df4cff9aec118a80931d53)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:30.325350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:30.327520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 92/176] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:30 2022] GET /identity/v3/users/b406de2279df4cff9aec118a80931d53 => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:30.334313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:30.425473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:30.451445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeceff-6100-41a4-828b-0a75aadd0ded None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:30.532055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320abe92-e649-4081-9b3d-fc8330198474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:30.533070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320abe92-e649-4081-9b3d-fc8330198474 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.533536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320abe92-e649-4081-9b3d-fc8330198474 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.534021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320abe92-e649-4081-9b3d-fc8330198474 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.535334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320abe92-e649-4081-9b3d-fc8330198474 None admin] RBAC: Adding query filter params (role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=a107bd9da4da4971b4c66f789a21033b) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:30.535661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320abe92-e649-4081-9b3d-fc8330198474 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:30.537162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320abe92-e649-4081-9b3d-fc8330198474 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:30.544298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 85/177] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:30 2022] GET /identity/v3/role_assignments?user.id=a107bd9da4da4971b4c66f789a21033b&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:30.545954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f1fd-77c9-44ee-9f8a-e0ab3eeba2ad None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.553015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f1fd-77c9-44ee-9f8a-e0ab3eeba2ad None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.553761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f1fd-77c9-44ee-9f8a-e0ab3eeba2ad None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.555748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 86/178] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:30.562699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18601a5-866f-4325-8de9-86d1f23244b2 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.563322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a130-e7eb-44b9-b402-be15b342e142 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:30.563564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18601a5-866f-4325-8de9-86d1f23244b2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.564175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18601a5-866f-4325-8de9-86d1f23244b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.597454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b18601a5-866f-4325-8de9-86d1f23244b2 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:30.599024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b18601a5-866f-4325-8de9-86d1f23244b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:30.634973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:30.635660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.635943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.636224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.669801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:30.747211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 87/179] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:30.758525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ab7c-bdee-41cb-be13-adfd79c3df18 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.759063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ab7c-bdee-41cb-be13-adfd79c3df18 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.759594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ab7c-bdee-41cb-be13-adfd79c3df18 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.761880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 88/180] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:30.763515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a03390-139e-4659-971d-35c4086f7498 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.763991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a03390-139e-4659-971d-35c4086f7498 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.764528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a03390-139e-4659-971d-35c4086f7498 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.766290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 89/181] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:30.771277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40535c1f-eb9d-4b4c-823b-dd787eea5441 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.771792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40535c1f-eb9d-4b4c-823b-dd787eea5441 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.772348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40535c1f-eb9d-4b4c-823b-dd787eea5441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.782133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:30.783884 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:30.807328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-40535c1f-eb9d-4b4c-823b-dd787eea5441 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:30.810211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-40535c1f-eb9d-4b4c-823b-dd787eea5441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:30.904644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 90/182] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:30.907425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ff45-b2ec-4ed5-b981-7f813747a5d1 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:30.908164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ff45-b2ec-4ed5-b981-7f813747a5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:30.908853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ff45-b2ec-4ed5-b981-7f813747a5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:30.932072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cf20ff45-b2ec-4ed5-b981-7f813747a5d1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:30.933327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cf20ff45-b2ec-4ed5-b981-7f813747a5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:30.935756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:30.937519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 93/183] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:30.938528 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:30.989130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:31.033088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 91/184] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:31.033088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb5d63-a555-4fc1-b837-e7d677593948 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.033088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb5d63-a555-4fc1-b837-e7d677593948 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.033088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb5d63-a555-4fc1-b837-e7d677593948 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.033088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 92/185] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:31.033088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb5d63-a555-4fc1-b837-e7d677593948 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.059183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f1be2-3cb3-4178-b42b-c63aa80fa8b8 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.100365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.101224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.101513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.101789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.114847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:31.123542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:31.125388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 94/186] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:48:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:31.126774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.187507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb5d63-a555-4fc1-b837-e7d677593948 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.225744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb71c7-e243-4dff-bcdd-d89be6f81be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.226668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb71c7-e243-4dff-bcdd-d89be6f81be4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.227050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb71c7-e243-4dff-bcdd-d89be6f81be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.227411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb71c7-e243-4dff-bcdd-d89be6f81be4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.229915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb71c7-e243-4dff-bcdd-d89be6f81be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:31.231553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb71c7-e243-4dff-bcdd-d89be6f81be4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:31.233464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 93/187] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:31.235561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968236-88a7-4498-902c-4803765393e1 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.236050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968236-88a7-4498-902c-4803765393e1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.236545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968236-88a7-4498-902c-4803765393e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.250845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a3d62-9ca3-4df8-83cf-ff925234c0ca None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.252249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:31.256840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:31.259904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-aa968236-88a7-4498-902c-4803765393e1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:31.261992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-aa968236-88a7-4498-902c-4803765393e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:31.317623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.319122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.319600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.320103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.322943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:31.325822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:31.329361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 95/188] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:31 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:31.330656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.353343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 94/189] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:31.355415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968236-88a7-4498-902c-4803765393e1 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.482180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7105-bf01-4040-a7ab-62d4f908380b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.520818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968236-88a7-4498-902c-4803765393e1 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.528697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.529576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.529862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.530142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.543357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:31.552232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:31.553256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:31.554136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:31.554740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 96/190] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:31.560918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.570954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.572468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.573040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.573471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.574545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:31.575040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:31.577540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:31.581474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:31.591657 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 95/191] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:48:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:31.599607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.611555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:31.709481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d475d-c825-4410-b04d-696e94626a20 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.727788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1653c-73ce-4e07-bde9-5aa63282547b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.753111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.753955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.754327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.754691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.755547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:31.757201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:31.768228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.768935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.769481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.769683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.770543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=b406de2279df4cff9aec118a80931d53) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:31.770917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:31.772448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:31.780538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 96/192] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:31 2022] GET /identity/v3/role_assignments?user.id=b406de2279df4cff9aec118a80931d53&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:31.793201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 97/193] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:48:31 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:31.793437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.793519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:31.929241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:31.929738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:31.947990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a9fb2-a1cd-415d-a9f4-efee85039084 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.992193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f82fa9-186a-455c-86c8-d7001b26362e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:31.992504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:31.993219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:31.993571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:31.994589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:31.995744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:31.996129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:31.997763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:32.008539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 98/194] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:32.009880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:32.067755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:32.068886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:32.069371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:32.070671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:32.090997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:32.099023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:32.100321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 97/195] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:32.102742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:32.210834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562364f-acda-49e1-8b9a-b6b648d9dc3b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:32.268133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:32.270534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:32.271179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:32.271845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] PATH_INFO: `/v3/users/088fe06bf1144abe92dba60295c5c10e` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:32.293987 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f22dc-4d88-4c95-88be-2136fa1696ca None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:32.299197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] RBAC: Authorizing `identity:get_user(user_id=088fe06bf1144abe92dba60295c5c10e)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:32.301971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:32.305176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 99/196] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:32 2022] GET /identity/v3/users/088fe06bf1144abe92dba60295c5c10e => generated 283 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:32.309629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:32.326662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:32.329807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:32.351426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:32.352814 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:32.353463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:32.353962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] PATH_INFO: `/v3/users/088fe06bf1144abe92dba60295c5c10e` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:32.356499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] RBAC: Authorizing `identity:get_user(user_id=088fe06bf1144abe92dba60295c5c10e)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:32.358831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:32.361321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 98/197] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:32 2022] GET /identity/v3/users/088fe06bf1144abe92dba60295c5c10e => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:32.363180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:32.510611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c51a0-eb68-47dc-8553-fdf7c48d7363 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:32.526583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa294c6-32e0-43db-9f89-d00fd3ab8d1a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:32.548352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:32.549463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:32.549746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:32.550110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:32.591786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:32.593006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:32.593608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:32.594186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:32.595698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:32.596509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:32.599359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:32.617417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 100/198] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:32.619776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:32.621680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:32.635842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:32.637480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:32.755282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:32.763965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 99/199] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:32.764658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:32.821602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554963cf-53b6-4243-8d80-e1aaea90fc38 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:32.901006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:32.902231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:32.902462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:32.903027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:32.903620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:32.912909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b1d5b-73ac-4205-895b-2b08821c1157 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:32.958372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:32.959290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:32.959647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:32.959929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] PATH_INFO: `/v3/users/c4e1a26d9c194e1f9b065b13134fc530` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:32.961307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=c4e1a26d9c194e1f9b065b13134fc530)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:32.963746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:32.965623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 100/200] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:32 2022] GET /identity/v3/users/c4e1a26d9c194e1f9b065b13134fc530 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:32.966355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:33.039406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:33.042303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:33.076874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae0224-9a3c-49d1-916c-75932068a7b7 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:33.114546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:33.115415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:33.115700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:33.115972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:33.117304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:33.119376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:33.121301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 101/201] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:32 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:33.122090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:33.157142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:33.167921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:33.226882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 101/202] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:33.229197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:33.240091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a59e-2934-43e8-b56c-dde6036d889f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:33.285470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:33.286826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:33.287277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:33.287688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:33.379502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:33.382228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:33.389696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147954-3272-45bd-80f7-153d16d382cf None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:33.422287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:33.430338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:33.431453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:33.431995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:33.432688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:33.433905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] RBAC: Adding query filter params (role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=088fe06bf1144abe92dba60295c5c10e) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:33.434443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:33.436190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:33.442856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 102/203] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:33 2022] GET /identity/v3/role_assignments?user.id=088fe06bf1144abe92dba60295c5c10e&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:33.444128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:33.488284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:33.545667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 102/204] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:33.563726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:33.661495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada60922-b92a-40c4-b4b2-e6d3fe0ce85a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:33.712873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:33.714168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:33.714663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:33.716534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:33.718136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:33.719825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:33.730175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 103/205] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:33 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:33.735918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:33.812499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677d475-9246-4433-8c42-aba3d1649777 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:33.870972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:33.891779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:33.893145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:33.894902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:33.895390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:33.901196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:33.907411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:33.909897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 103/206] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:33 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:33.919616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:33.964068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b67fa9-701d-479e-bb5b-c8732d3e3087 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:33.965798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:34.042352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc82b4-adc0-4a4c-8972-425748241222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:34.043044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc82b4-adc0-4a4c-8972-425748241222 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:34.043334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc82b4-adc0-4a4c-8972-425748241222 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:34.043618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc82b4-adc0-4a4c-8972-425748241222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:34.044476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc82b4-adc0-4a4c-8972-425748241222 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=088fe06bf1144abe92dba60295c5c10e) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:34.044796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc82b4-adc0-4a4c-8972-425748241222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:34.046299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc82b4-adc0-4a4c-8972-425748241222 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:34.062548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 104/207] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:33 2022] GET /identity/v3/role_assignments?user.id=088fe06bf1144abe92dba60295c5c10e&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:34.148602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2b8ff-e485-4731-ba6c-fcdc0336e939 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:34.221143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17be98-8cb5-438f-acc9-4e4a681f7799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:34.222829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17be98-8cb5-438f-acc9-4e4a681f7799 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:34.223658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17be98-8cb5-438f-acc9-4e4a681f7799 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:34.224444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17be98-8cb5-438f-acc9-4e4a681f7799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:34.226571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17be98-8cb5-438f-acc9-4e4a681f7799 None admin] RBAC: Adding query filter params (role.id=00ac4a79282e459f8e0aa6c551f0322d, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc, user.id=c4e1a26d9c194e1f9b065b13134fc530) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:34.227539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17be98-8cb5-438f-acc9-4e4a681f7799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:34.230706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17be98-8cb5-438f-acc9-4e4a681f7799 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:34.238458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 104/208] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:33 2022] GET /identity/v3/role_assignments?user.id=c4e1a26d9c194e1f9b065b13134fc530&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=00ac4a79282e459f8e0aa6c551f0322d => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:34.636741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8695f4-4237-4a79-a4a2-e9fcbce509fc None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:34.637821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8695f4-4237-4a79-a4a2-e9fcbce509fc None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:34.638307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8695f4-4237-4a79-a4a2-e9fcbce509fc None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:34.640945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 105/209] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:34.649062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67c4c0-85ba-4102-9c70-bc0be103fd60 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:34.650888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67c4c0-85ba-4102-9c70-bc0be103fd60 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:34.651673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67c4c0-85ba-4102-9c70-bc0be103fd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:34.693210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3a67c4c0-85ba-4102-9c70-bc0be103fd60 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:34.695501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3a67c4c0-85ba-4102-9c70-bc0be103fd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:34.784179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:34.803683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 105/210] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:34.856913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8695f4-4237-4a79-a4a2-e9fcbce509fc None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:34.965220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93dde2-18be-450b-bce5-3b63865ed9a0 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:34.965889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93dde2-18be-450b-bce5-3b63865ed9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:34.966367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93dde2-18be-450b-bce5-3b63865ed9a0 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:34.968039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 106/211] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:34.974374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6616c-ccdc-4da4-8583-db420a9637b9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:34.974972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6616c-ccdc-4da4-8583-db420a9637b9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:34.975470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6616c-ccdc-4da4-8583-db420a9637b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:34.994696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a0c6616c-ccdc-4da4-8583-db420a9637b9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:34.996579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a0c6616c-ccdc-4da4-8583-db420a9637b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:35.026930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8695f4-4237-4a79-a4a2-e9fcbce509fc None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:35.077554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:35.078238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:35.078532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:35.078808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:35.091434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:35.093265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 107/212] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:35.100099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:35.101784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:35.103252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 106/213] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:35.115073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:35.132879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6616c-ccdc-4da4-8583-db420a9637b9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:35.175614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:35.246565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6616c-ccdc-4da4-8583-db420a9637b9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:35.266626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:35.286712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:35.287770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:35.288203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:35.288661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:35.290104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:35.292263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:35.294216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 108/214] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:35.300804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:35.323287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5462dd-3295-4fa5-837c-96df7e766a89 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:35.365160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:35.366186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:35.366809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:35.367375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:35.368354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:35.369444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:35.371215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:35.381733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 107/215] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:35.395344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:35.422910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567f6a-3a87-4be2-9385-73699d11c8cb None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:35.461175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:35.461871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:35.462157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:35.462471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:35.463631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:35.465171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:35.497043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 109/216] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:48:35 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:35.562288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:35.578210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c9f69-0624-4bf3-af2d-d10723b1b0c8 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:35.618541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:35.619377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:35.619658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:35.619937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] PATH_INFO: `/v3/users/2713c3c9da564634913e06a92a482192` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:35.637251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] RBAC: Authorizing `identity:get_user(user_id=2713c3c9da564634913e06a92a482192)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:35.638827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:35.640967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 108/217] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:35 2022] GET /identity/v3/users/2713c3c9da564634913e06a92a482192 => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:35.646867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:35.702654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:35.774188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a3e12-d0c3-4d13-993a-4fa88d775ffb None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:35.814244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:35.815158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:35.815593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:35.816017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:35.922928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:35.924689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:35.992757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:36.083288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 110/218] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:48:36.093668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:36.338215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b8757-b98c-47df-914c-b220ad70bc66 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:36.400660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e304e-960d-434f-8cbb-537413b99fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:36.401570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e304e-960d-434f-8cbb-537413b99fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:36.401855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e304e-960d-434f-8cbb-537413b99fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:36.402901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e304e-960d-434f-8cbb-537413b99fa5 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:36.409911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e304e-960d-434f-8cbb-537413b99fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:36.411418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e304e-960d-434f-8cbb-537413b99fa5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:36.413387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 109/219] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:36 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:36.424186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:36.428442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24446c29-89ae-45df-9dd4-ee1cf3732d99 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:36.434521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24446c29-89ae-45df-9dd4-ee1cf3732d99 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:36.434916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24446c29-89ae-45df-9dd4-ee1cf3732d99 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:36.437203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 110/220] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:36.446802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c443d79-0001-4aa8-a826-85c0b2a8a7b1 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:36.448700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c443d79-0001-4aa8-a826-85c0b2a8a7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:36.449542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c443d79-0001-4aa8-a826-85c0b2a8a7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:36.497179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-1c443d79-0001-4aa8-a826-85c0b2a8a7b1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:36.498831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1c443d79-0001-4aa8-a826-85c0b2a8a7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:36.529375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:36.622735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8156d8-9db7-43de-9446-172cb91b4a32 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:36.678751 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 111/221] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:36.694619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:36.695295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:36.695582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:36.695859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:36.697313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=2713c3c9da564634913e06a92a482192) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:36.697599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:36.699238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:36.709934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 111/222] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:36 2022] GET /identity/v3/role_assignments?user.id=2713c3c9da564634913e06a92a482192&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=acffa214db594ed9803f8f5376b15d42 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:36.719681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c443d79-0001-4aa8-a826-85c0b2a8a7b1 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:36.842165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c443d79-0001-4aa8-a826-85c0b2a8a7b1 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:36.886578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62a15c-76dc-4295-86df-fb097ba70c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:36.887376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a15c-76dc-4295-86df-fb097ba70c53 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:36.887657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a15c-76dc-4295-86df-fb097ba70c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:36.887930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a15c-76dc-4295-86df-fb097ba70c53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:36.902164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62a15c-76dc-4295-86df-fb097ba70c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-9a62a15c-76dc-4295-86df-fb097ba70c53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:36.910875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:36.911834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:36.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:36.917008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 112/223] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:48:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:36.919246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:36.929998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:36.965389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25ae78-ba14-4ac3-bd20-2ae8a6931508 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:36.966194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25ae78-ba14-4ac3-bd20-2ae8a6931508 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:36.966819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25ae78-ba14-4ac3-bd20-2ae8a6931508 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:36.968775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 113/224] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:36.978934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a73891-0802-487b-80d3-ae0ca3398340 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:36.979547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a73891-0802-487b-80d3-ae0ca3398340 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:36.980211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a73891-0802-487b-80d3-ae0ca3398340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:36.989768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:37.013662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-10a73891-0802-487b-80d3-ae0ca3398340 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:37.017727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-10a73891-0802-487b-80d3-ae0ca3398340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:37.066642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613a7ce-dd54-4f2d-b9d6-a6863e55cd1f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:37.134854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:37.135554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:37.136097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:37.136348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:37.137201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:37.137444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:37.138838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:37.144803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 114/225] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:37.161732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 112/226] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:48:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:37.167019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a73891-0802-487b-80d3-ae0ca3398340 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:37.179676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:37.395514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:37.435251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a73891-0802-487b-80d3-ae0ca3398340 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:37.454345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe18b3-a35a-4c21-87cf-322ecb7d1b7d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:37.511039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:37.513068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:37.513786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:37.514257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] PATH_INFO: `/v3/users/088fe06bf1144abe92dba60295c5c10e` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:37.516149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] RBAC: Authorizing `identity:get_user(user_id=088fe06bf1144abe92dba60295c5c10e)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:37.518778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:37.522346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 115/227] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:37 2022] GET /identity/v3/users/088fe06bf1144abe92dba60295c5c10e => generated 283 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:37.532177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:37.545635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:37.570807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:37.574256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:37.574772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:37.575222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:37.599460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:37.617615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:37.619942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:37.620897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:37.629386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 113/228] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:37.632946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:37.730810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576fa93-b370-440b-b9b9-f5c54991e136 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:37.783358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:37.784282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:37.784577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:37.784862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:37.786310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:37.787827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:37.789815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 116/229] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:37 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:37.800270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:37.829252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972db789-99f0-4041-8b7a-aaec71313d37 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:37.870288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:37.883663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:37.886333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:37.886885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:37.887934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:37.889278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:37.889748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:37.891725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:37.896930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:37.910426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 114/230] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:37.919422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:37.937927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fdf79-e7b7-406f-a9f8-795e67f4e723 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:37.984813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:37.986118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:37.986414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:37.987523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/088fe06bf1144abe92dba60295c5c10e/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:37.990574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] RBAC: Authorizing `identity:create_grant(project_id=114512b9b8fe40f4a5d4d34ea099a4ba, user_id=088fe06bf1144abe92dba60295c5c10e, role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:37.993238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:38.018364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 117/231] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:37 2022] PUT /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/088fe06bf1144abe92dba60295c5c10e/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:48:38.115715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c2843-e39c-46e8-ad17-b3cb18389805 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:38.158684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:38.159599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:38.159887 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:38.160168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] PATH_INFO: `/v3/users/088fe06bf1144abe92dba60295c5c10e` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:38.161607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] RBAC: Authorizing `identity:get_user(user_id=088fe06bf1144abe92dba60295c5c10e)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:38.163183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:38.165431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 115/232] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:37 2022] GET /identity/v3/users/088fe06bf1144abe92dba60295c5c10e => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:38.174263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:38.319745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d0bc-9cc9-4065-b49b-d5724423ebfb None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:38.364265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:38.373277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f24b9-ccc1-4085-bb70-bde53b490e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:38.374538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f24b9-ccc1-4085-bb70-bde53b490e40 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:38.377536 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f24b9-ccc1-4085-bb70-bde53b490e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:38.377871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f24b9-ccc1-4085-bb70-bde53b490e40 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:38.380262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f24b9-ccc1-4085-bb70-bde53b490e40 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:38.380919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f24b9-ccc1-4085-bb70-bde53b490e40 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:38.382938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 118/233] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:38 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:38.389552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:38.519672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:38.579690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e3993-6642-42c1-a5e8-5a5810f491db None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:38.579986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e3993-6642-42c1-a5e8-5a5810f491db None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:38.580211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e3993-6642-42c1-a5e8-5a5810f491db None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:38.581692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 119/234] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:38.589268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae905ec-f709-4776-b11c-b13e6af54e8d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:38.589609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae905ec-f709-4776-b11c-b13e6af54e8d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:38.589840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae905ec-f709-4776-b11c-b13e6af54e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:38.606018 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2600e8-ba4e-44cc-b807-527662510efd None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:38.615086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-dae905ec-f709-4776-b11c-b13e6af54e8d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:38.616320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-dae905ec-f709-4776-b11c-b13e6af54e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:38.663346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:38.664330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:38.664618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:38.664900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/088fe06bf1144abe92dba60295c5c10e/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:38.667639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=114512b9b8fe40f4a5d4d34ea099a4ba, user_id=088fe06bf1144abe92dba60295c5c10e, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:38.669156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:38.694508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 116/235] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:38 2022] PUT /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/088fe06bf1144abe92dba60295c5c10e/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:38.744817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 120/236] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:38.787037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:38.898673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7610d4-c26d-4ef4-a3b1-297dc0b78494 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:38.898883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7610d4-c26d-4ef4-a3b1-297dc0b78494 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:38.899274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7610d4-c26d-4ef4-a3b1-297dc0b78494 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:38.901605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 121/237] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:38.907435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d915a-1d04-434e-a54d-53f854a5cb70 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:38.907745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d915a-1d04-434e-a54d-53f854a5cb70 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:38.908007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d915a-1d04-434e-a54d-53f854a5cb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:38.927430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-1a8d915a-1d04-434e-a54d-53f854a5cb70 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:38.928713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1a8d915a-1d04-434e-a54d-53f854a5cb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:38.971134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b84035-027d-4a71-baa8-cb4d1dbcf7d7 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:39.005057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:39.066142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 122/238] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:39.078532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:39.079970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:39.080456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:39.080935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:39.083294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:39.086512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:39.091036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d915a-1d04-434e-a54d-53f854a5cb70 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:39.091542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 117/239] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:39.100492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:39.192654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:39.285407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d915a-1d04-434e-a54d-53f854a5cb70 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:39.299993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e90b4e-c99c-4cfe-a59a-96ccd4f39586 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:39.342713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:39.343534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:39.343817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:39.344101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:39.357327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:39.366001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:39.367758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:39.369487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 123/240] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:39.384923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:39.442213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:39.443953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:39.444691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:39.445480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:39.447665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:39.450899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:39.520453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 118/241] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:48:39 2022] POST /identity/v3/users => generated 293 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:39.527371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5417f-3016-456e-9018-d0150e14627f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:39.551327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:39.575023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:39.575748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:39.576038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:39.576780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:39.577612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:39.577932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:39.579398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:39.591950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 124/242] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:39.597624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:39.661905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:39.762649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576741d9-4d80-45f5-b2d3-80378667e70e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:39.815179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:39.816134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:39.816464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:39.816752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] PATH_INFO: `/v3/users/b7da97f4918b4e518965c2d73518969d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:39.839295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] RBAC: Authorizing `identity:get_user(user_id=b7da97f4918b4e518965c2d73518969d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:39.840861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:39.842741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 119/243] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:39 2022] GET /identity/v3/users/b7da97f4918b4e518965c2d73518969d => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:39.848767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:39.970614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18e80a-3c17-4015-81a2-c749bdaaf7e0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:40.017062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:40.018076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:40.018542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:40.018993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:40.075450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:40.101169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:40.103615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:40.314029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 125/244] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:40.320185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:40.542678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:40.549603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19916e-e825-4519-a32d-3e5297b3561b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:40.626156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2576fa-0ea0-476a-a555-1db8f4302e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:40.627582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2576fa-0ea0-476a-a555-1db8f4302e93 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:40.628078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2576fa-0ea0-476a-a555-1db8f4302e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:40.628744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2576fa-0ea0-476a-a555-1db8f4302e93 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:40.635142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2576fa-0ea0-476a-a555-1db8f4302e93 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:40.637889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2576fa-0ea0-476a-a555-1db8f4302e93 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:40.641690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 120/245] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:40 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:40.651517 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:40.841356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:40.894390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0da6b-fe50-46c1-ac36-f76eab7c7351 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:40.944801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d42ba-95f1-4416-8fde-fe4672ca457a None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:40.945313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d42ba-95f1-4416-8fde-fe4672ca457a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:40.945848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d42ba-95f1-4416-8fde-fe4672ca457a None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:40.957034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 121/246] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:40.973638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b0e56-8f4b-4bc8-809e-7595a045ba5d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:40.974147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b0e56-8f4b-4bc8-809e-7595a045ba5d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:40.974647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b0e56-8f4b-4bc8-809e-7595a045ba5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:41.031256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b63b0e56-8f4b-4bc8-809e-7595a045ba5d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:41.033409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b63b0e56-8f4b-4bc8-809e-7595a045ba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:41.034174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:41.035299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:41.035801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:41.036292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:41.037874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] RBAC: Adding query filter params (role.id=00ac4a79282e459f8e0aa6c551f0322d, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=b7da97f4918b4e518965c2d73518969d) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:41.038423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:41.041771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:41.076201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 126/247] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:40 2022] GET /identity/v3/role_assignments?user.id=b7da97f4918b4e518965c2d73518969d&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=00ac4a79282e459f8e0aa6c551f0322d => generated 259 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:41.241046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 122/248] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:41.285514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:41.463794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82f5f2-7cd0-4c75-89ca-5cd4f280177a None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:41.464566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82f5f2-7cd0-4c75-89ca-5cd4f280177a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:41.464566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82f5f2-7cd0-4c75-89ca-5cd4f280177a None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:41.469145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 123/249] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:41.475484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b558c47-beb1-4e44-b5b5-e262ae3ea444 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:41.479170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450911d-84d9-460c-8f1e-9c658df1f9ee None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:41.479673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450911d-84d9-460c-8f1e-9c658df1f9ee None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:41.479817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450911d-84d9-460c-8f1e-9c658df1f9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:41.529069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-d450911d-84d9-460c-8f1e-9c658df1f9ee None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:41.530302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d450911d-84d9-460c-8f1e-9c658df1f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:41.598658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:41.600536 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:41.601477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:41.602271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:41.626132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:41.645177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:41.647001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:41.648464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:41.653978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 127/250] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:41.666991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:41.702659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:41.790826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 124/251] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:41.831305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450911d-84d9-460c-8f1e-9c658df1f9ee None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:41.934287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:42.004874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0cb39-b047-475b-a7e9-afe4a791be51 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:42.133712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f785b0b-65cf-47fc-af36-a6c3668dc39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:42.136470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f785b0b-65cf-47fc-af36-a6c3668dc39c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:42.137306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f785b0b-65cf-47fc-af36-a6c3668dc39c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:42.138088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f785b0b-65cf-47fc-af36-a6c3668dc39c None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:42.138264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450911d-84d9-460c-8f1e-9c658df1f9ee None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:42.139960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f785b0b-65cf-47fc-af36-a6c3668dc39c None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:42.140807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f785b0b-65cf-47fc-af36-a6c3668dc39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:42.149007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f785b0b-65cf-47fc-af36-a6c3668dc39c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:42.182694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 128/252] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 518 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:42.185767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e86e4-00c3-41b0-b58d-8caa844cf91d None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:42.186622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e86e4-00c3-41b0-b58d-8caa844cf91d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:42.187429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e86e4-00c3-41b0-b58d-8caa844cf91d None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:42.190460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 129/253] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:42.195227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e86e4-00c3-41b0-b58d-8caa844cf91d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:42.228845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595ca7e-1ff3-4cfe-a215-13c2706c0a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:42.230328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595ca7e-1ff3-4cfe-a215-13c2706c0a35 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:42.230803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595ca7e-1ff3-4cfe-a215-13c2706c0a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:42.233072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595ca7e-1ff3-4cfe-a215-13c2706c0a35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:42.278526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595ca7e-1ff3-4cfe-a215-13c2706c0a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-a595ca7e-1ff3-4cfe-a215-13c2706c0a35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:42.308544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:42.310317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:42.311842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:42.315597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 125/254] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:48:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:42.327543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc6774-f9ef-46a0-9361-8b6ddfbc5ef4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:42.328052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc6774-f9ef-46a0-9361-8b6ddfbc5ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:42.328551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc6774-f9ef-46a0-9361-8b6ddfbc5ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:42.385181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a1bc6774-f9ef-46a0-9361-8b6ddfbc5ef4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:42.386941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a1bc6774-f9ef-46a0-9361-8b6ddfbc5ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:42.437308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:42.538594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 126/255] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:42.541950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc6774-f9ef-46a0-9361-8b6ddfbc5ef4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:42.573743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e86e4-00c3-41b0-b58d-8caa844cf91d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:42.583085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:42.671130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:42.671931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:42.672223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:42.672504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] PATH_INFO: `/v3/users/2713c3c9da564634913e06a92a482192` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:42.677832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] RBAC: Authorizing `identity:get_user(user_id=2713c3c9da564634913e06a92a482192)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:42.679365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:42.682130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 130/256] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:42 2022] GET /identity/v3/users/2713c3c9da564634913e06a92a482192 => generated 297 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:42.682948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:42.764394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc6774-f9ef-46a0-9361-8b6ddfbc5ef4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:42.839385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:42.841241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:42.842026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:42.842794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:42.844914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:42.845840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:42.848932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:42.866717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 127/257] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:48:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:42.869476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:42.965459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05047a15-8dfa-4b41-bd07-dd40e02daebf None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:43.042201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:43.085771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:43.086586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:43.086948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:43.086948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:43.089364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713021-c0c1-4e84-aae3-d613e9afd913 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:43.090090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:43.118043 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:43.136039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:43.138512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:43.140132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 131/258] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:48:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:43.143365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:43.175526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:43.177763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:43.178606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:43.179448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:43.182193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:43.185144 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:43.189385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 128/259] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:42 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:43.191251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:43.409161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a30d-46eb-4163-bdb5-480eaaede16f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:43.479639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450062c4-db4e-48ff-9620-a0c67492b923 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:43.485168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:43.486797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:43.487589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:43.488359 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:43.490294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:43.491222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:43.495522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:43.515164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 129/260] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:43.517343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:43.547866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:43.633728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:43.635247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:43.635537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:43.635818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] PATH_INFO: `/v3/users/088fe06bf1144abe92dba60295c5c10e` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:43.641877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] RBAC: Authorizing `identity:get_user(user_id=088fe06bf1144abe92dba60295c5c10e)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:43.643395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:43.645573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 132/261] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:43 2022] GET /identity/v3/users/088fe06bf1144abe92dba60295c5c10e => generated 283 bytes in 507 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:43.646485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:43.745257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab7cdd-8475-4797-b459-64a9bcaf71ab None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:43.793875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:43.816196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:43.817908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:43.819020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:43.819534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/2713c3c9da564634913e06a92a482192/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:43.825067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] RBAC: Authorizing `identity:create_grant(project_id=114512b9b8fe40f4a5d4d34ea099a4ba, user_id=2713c3c9da564634913e06a92a482192, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:43.828473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:43.848967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa619714-ae8b-4a6d-8417-0c50c933a4ac None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:43.865108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 130/262] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:43 2022] PUT /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/2713c3c9da564634913e06a92a482192/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:43.867098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:43.956689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01515a86-261e-403c-80a6-87f3afbd909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:43.958520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515a86-261e-403c-80a6-87f3afbd909b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:43.959274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515a86-261e-403c-80a6-87f3afbd909b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:43.960050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515a86-261e-403c-80a6-87f3afbd909b None admin] PATH_INFO: `/v3/users/088fe06bf1144abe92dba60295c5c10e` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:43.963537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01515a86-261e-403c-80a6-87f3afbd909b None admin] RBAC: Authorizing `identity:get_user(user_id=088fe06bf1144abe92dba60295c5c10e)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:43.966539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01515a86-261e-403c-80a6-87f3afbd909b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:43.976716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 133/263] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:43 2022] GET /identity/v3/users/088fe06bf1144abe92dba60295c5c10e => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:43.980152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7b24-e817-471b-8dc2-d8f611335061 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:43.981047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7b24-e817-471b-8dc2-d8f611335061 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:43.981770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7b24-e817-471b-8dc2-d8f611335061 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:43.985547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 134/264] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:43.987686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d7b24-e817-471b-8dc2-d8f611335061 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:44.081036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:44.193178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641ad26-d19e-4aa0-8526-2f64f0ac1a6b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:44.285906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e7dbc-bc52-4c0e-b3f9-0471f872030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:44.287322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e7dbc-bc52-4c0e-b3f9-0471f872030d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:44.288004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e7dbc-bc52-4c0e-b3f9-0471f872030d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:44.288819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e7dbc-bc52-4c0e-b3f9-0471f872030d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:44.422483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d7b24-e817-471b-8dc2-d8f611335061 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:44.516464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:44.527040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e7dbc-bc52-4c0e-b3f9-0471f872030d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:44.530768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e7dbc-bc52-4c0e-b3f9-0471f872030d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:44.530962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2561cc-ebdc-4f1d-bfe3-e229443e6f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:44.531908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2561cc-ebdc-4f1d-bfe3-e229443e6f6f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:44.532399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2561cc-ebdc-4f1d-bfe3-e229443e6f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:44.532899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2561cc-ebdc-4f1d-bfe3-e229443e6f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:44.673906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2561cc-ebdc-4f1d-bfe3-e229443e6f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:44.677174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2561cc-ebdc-4f1d-bfe3-e229443e6f6f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:44.691722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:44.769033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 131/265] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 903 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:44.771489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a5fca-72b1-4e21-8b18-da186b3e119b None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:44.771992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a5fca-72b1-4e21-8b18-da186b3e119b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:44.772485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a5fca-72b1-4e21-8b18-da186b3e119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:44.801540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-968a5fca-72b1-4e21-8b18-da186b3e119b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:44.803726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-968a5fca-72b1-4e21-8b18-da186b3e119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:44.895343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 135/266] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:44.898131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5be8bc-3f08-48e0-ae3d-51f6a02fe111 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:44.898620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5be8bc-3f08-48e0-ae3d-51f6a02fe111 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:44.899105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5be8bc-3f08-48e0-ae3d-51f6a02fe111 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:44.902410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 136/267] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:44.906653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec176df-92a5-4363-9990-f353840e660a None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:44.906999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec176df-92a5-4363-9990-f353840e660a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:44.907260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec176df-92a5-4363-9990-f353840e660a None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:44.909174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 137/268] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:44.909947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec176df-92a5-4363-9990-f353840e660a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:44.943898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 132/269] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:44.944815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:44.948233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a5fca-72b1-4e21-8b18-da186b3e119b None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:45.081561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec176df-92a5-4363-9990-f353840e660a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:45.122596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:45.128486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a5fca-72b1-4e21-8b18-da186b3e119b None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:45.146760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff31313-0ebc-48b1-8647-815c52706424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:45.148576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31313-0ebc-48b1-8647-815c52706424 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.149083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31313-0ebc-48b1-8647-815c52706424 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.150118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31313-0ebc-48b1-8647-815c52706424 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.152495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff31313-0ebc-48b1-8647-815c52706424 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:45.154964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff31313-0ebc-48b1-8647-815c52706424 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:45.157915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 138/270] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:44 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:45.159801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9debf7b-138a-4a9e-9d3c-fecba19e4671 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.160193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9debf7b-138a-4a9e-9d3c-fecba19e4671 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.160591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9debf7b-138a-4a9e-9d3c-fecba19e4671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.186724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f9debf7b-138a-4a9e-9d3c-fecba19e4671 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:45.188874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f9debf7b-138a-4a9e-9d3c-fecba19e4671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:45.191301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1d22e-ecb4-4b03-a414-d1dcc00f9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:45.192435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1d22e-ecb4-4b03-a414-d1dcc00f9946 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.192891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1d22e-ecb4-4b03-a414-d1dcc00f9946 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.193350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1d22e-ecb4-4b03-a414-d1dcc00f9946 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.195584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1d22e-ecb4-4b03-a414-d1dcc00f9946 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:45.197321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1d22e-ecb4-4b03-a414-d1dcc00f9946 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:45.199812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 133/271] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:44 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:45.203261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f553-2e2f-484f-be90-03c19fffec54 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.204065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f553-2e2f-484f-be90-03c19fffec54 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.204962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f553-2e2f-484f-be90-03c19fffec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.226513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1c12f553-2e2f-484f-be90-03c19fffec54 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:45.228814 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1c12f553-2e2f-484f-be90-03c19fffec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:45.325191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 134/272] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:45.325191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12f553-2e2f-484f-be90-03c19fffec54 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:45.341289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 139/273] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:45.343319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9debf7b-138a-4a9e-9d3c-fecba19e4671 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:45.404820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:45.433570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:45.438228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12f553-2e2f-484f-be90-03c19fffec54 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:45.478674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:45.479537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.479817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.480095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.499080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:45.507110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:45.508034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:45.509406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 135/274] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:45.510707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:45.551282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9debf7b-138a-4a9e-9d3c-fecba19e4671 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:45.601159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:45.601853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.602125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.602409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.603316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] RBAC: Adding query filter params (role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=088fe06bf1144abe92dba60295c5c10e) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:45.603637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:45.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:45.611764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 140/275] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:45 2022] GET /identity/v3/role_assignments?user.id=088fe06bf1144abe92dba60295c5c10e&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:45.621183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:45.677991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c018041-bab3-48a0-a626-6e9bfb09214b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:45.768270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:45.769743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.770235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.770831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.772759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=088fe06bf1144abe92dba60295c5c10e) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:45.773465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:45.777514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:45.789389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 136/276] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:45 2022] GET /identity/v3/role_assignments?user.id=088fe06bf1144abe92dba60295c5c10e&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:45.793404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:45.818531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:45.836683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:45.837678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46c41-c22c-4254-aaac-96cc195717c3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:45.914244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:45.916314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:45.916781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:45.918262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:45.919944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:45.923910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:45.929088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 141/277] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:45.933015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:45.947913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33cc7-d7b6-4ee6-9764-ddbab5490def None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:46.015553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:46.018396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:46.019895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:46.024578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:46.040994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:46.049687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:46.050632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:46.051536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:46.054399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 137/278] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:46.055509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:46.139575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fa388-956b-4bb8-9935-692b6e23bb0d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:46.216602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:46.224906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:46.226125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:46.226607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:46.227109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:46.229428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:46.230667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:46.233657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:46.257929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 142/279] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:46.258146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6479f-cad0-4f35-8acd-2448718c6045 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:46.258229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:46.300252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:46.301413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:46.301706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:46.301966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:46.302711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:46.304195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:46.308343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:46.335260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 138/280] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:48:46 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:46.338314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:46.463930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f13e1-1b7f-4881-8524-9d1d22553c59 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:46.520298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:46.521244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:46.521494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:46.521838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:46.522683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:46.523058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:46.525128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:46.535150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 143/281] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:46.541394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:46.544726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0eda-ab96-46a7-ab16-ee5990ac0359 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:46.590804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:46.591688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:46.591975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:46.592255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] PATH_INFO: `/v3/users/6ae0ba17410f40e890fe7b99e2fb46cc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:46.608230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:46.628318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] RBAC: Authorizing `identity:get_user(user_id=6ae0ba17410f40e890fe7b99e2fb46cc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:46.631107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:46.636361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 139/282] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:46 2022] GET /identity/v3/users/6ae0ba17410f40e890fe7b99e2fb46cc => generated 293 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:46.645322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:46.670471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:46.696893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48dced-6297-45f7-9be8-f0bbcfd9e085 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:46.767397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:46.768861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:46.769416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:46.769931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] PATH_INFO: `/v3/users/b7da97f4918b4e518965c2d73518969d` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:46.772186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] RBAC: Authorizing `identity:get_user(user_id=b7da97f4918b4e518965c2d73518969d)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:46.775356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:46.780284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 144/283] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:46 2022] GET /identity/v3/users/b7da97f4918b4e518965c2d73518969d => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:46.791252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:46.861743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea53b9-1770-4753-80fc-243d2e3b130d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:46.923036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:46.924104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:46.924572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:46.925071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:46.926533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, user.id=6ae0ba17410f40e890fe7b99e2fb46cc, scope.system=all) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:46.927086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:46.929852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:46.945007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 140/284] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jun 29 07:48:46 2022] GET /identity/v3/role_assignments?user.id=6ae0ba17410f40e890fe7b99e2fb46cc&scope.system=all&role.id=acffa214db594ed9803f8f5376b15d42 => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:46.981861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccbf1-b49c-4c29-a0f3-2ad5d2bf5022 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:47.074487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155f8e9-2671-4d5e-b073-e332ddce4824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:47.076056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155f8e9-2671-4d5e-b073-e332ddce4824 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:47.076579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155f8e9-2671-4d5e-b073-e332ddce4824 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:47.077120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155f8e9-2671-4d5e-b073-e332ddce4824 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:47.079609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155f8e9-2671-4d5e-b073-e332ddce4824 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:47.082559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155f8e9-2671-4d5e-b073-e332ddce4824 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:47.086238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 145/285] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:46 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:47.094711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:47.181507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7ede-b052-4296-be30-63bb67f023c7 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:47.181909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7ede-b052-4296-be30-63bb67f023c7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:47.182392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7ede-b052-4296-be30-63bb67f023c7 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:47.186382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 146/286] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:47.197113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4b83-b48e-455d-9c1a-42144951bf0f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:47.197656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4b83-b48e-455d-9c1a-42144951bf0f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:47.198159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4b83-b48e-455d-9c1a-42144951bf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:47.212715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:47.250423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d99a4b83-b48e-455d-9c1a-42144951bf0f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:47.252471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d99a4b83-b48e-455d-9c1a-42144951bf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:47.302340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a0f72-0da6-4cc7-8cc9-26561893abb3 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:47.323789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:47.404877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:47.406465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:47.406955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:47.407427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/b7da97f4918b4e518965c2d73518969d/roles/00ac4a79282e459f8e0aa6c551f0322d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:47.415835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] RBAC: Authorizing `identity:create_grant(project_id=114512b9b8fe40f4a5d4d34ea099a4ba, user_id=b7da97f4918b4e518965c2d73518969d, role_id=00ac4a79282e459f8e0aa6c551f0322d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:47.418927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:47.453984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 141/287] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:48:47 2022] PUT /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/users/b7da97f4918b4e518965c2d73518969d/roles/00ac4a79282e459f8e0aa6c551f0322d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:47.503422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 147/288] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:47.546963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:47.716389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ad3d-bb34-43d2-817d-8e775aca2c1f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:47.751111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:47.778445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:47.779919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:47.780209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:47.780751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:47.800321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:47.808662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:47.810538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:47.812133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 142/289] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:47.826876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4b83-b48e-455d-9c1a-42144951bf0f None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:47.959454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4b83-b48e-455d-9c1a-42144951bf0f None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:48.009968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:48.011005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:48.011483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:48.011970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:48.012931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:48.013470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:48.015248 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:48.029971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 148/290] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:48.035490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:48.176165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c548db7-2f9c-4351-91b5-5bc1bd14e798 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:48.216620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:48.217566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:48.217857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:48.218137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] PATH_INFO: `/v3/users/2713c3c9da564634913e06a92a482192` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:48.221782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] RBAC: Authorizing `identity:get_user(user_id=2713c3c9da564634913e06a92a482192)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:48.223292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:48.225102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 143/291] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:48 2022] GET /identity/v3/users/2713c3c9da564634913e06a92a482192 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:48.231440 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:48.273837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:48.402153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf277-e49d-406c-9b06-6208c8b0dd98 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:48.439684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:48.440354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:48.440641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:48.440925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:48.528891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:48.530376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:48.555767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:48.662847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 149/292] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:48:48.671383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:48.786646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:48.864759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8ef97-d5b5-4610-a669-99bf9061505f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:48.908505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aad774-2797-4d92-98cb-8493e7ca376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:48.909422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aad774-2797-4d92-98cb-8493e7ca376b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:48.909707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aad774-2797-4d92-98cb-8493e7ca376b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:48.909982 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aad774-2797-4d92-98cb-8493e7ca376b None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:48.911223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aad774-2797-4d92-98cb-8493e7ca376b None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:48.912686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aad774-2797-4d92-98cb-8493e7ca376b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:48.914975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 144/293] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:48 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:48.921872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:49.086265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f11d6-8e95-45e6-913b-df1bff6d1fdd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:49.126296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:49.179612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6874e-2762-4272-a123-9385a37c6561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:49.181737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6874e-2762-4272-a123-9385a37c6561 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:49.182238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6874e-2762-4272-a123-9385a37c6561 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:49.182729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6874e-2762-4272-a123-9385a37c6561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:49.184315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6874e-2762-4272-a123-9385a37c6561 None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=2713c3c9da564634913e06a92a482192) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:49.184871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6874e-2762-4272-a123-9385a37c6561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:49.187734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6874e-2762-4272-a123-9385a37c6561 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:49.203552 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 150/294] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:48 2022] GET /identity/v3/role_assignments?user.id=2713c3c9da564634913e06a92a482192&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=acffa214db594ed9803f8f5376b15d42 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:49.646039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b30a5-197d-4576-b66b-626af520d61e None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:49.647505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b30a5-197d-4576-b66b-626af520d61e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:49.648279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b30a5-197d-4576-b66b-626af520d61e None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:49.651803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 145/295] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:49.658565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81649781-f1ce-4d8e-8bf6-f0e686438735 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:49.659455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81649781-f1ce-4d8e-8bf6-f0e686438735 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:49.660228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81649781-f1ce-4d8e-8bf6-f0e686438735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:49.692054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-81649781-f1ce-4d8e-8bf6-f0e686438735 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:49.694529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-81649781-f1ce-4d8e-8bf6-f0e686438735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:49.840868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 151/296] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:49.869888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b30a5-197d-4576-b66b-626af520d61e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:49.911240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2c5da-5dcc-40c2-83bb-728c43e6508d None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:49.912107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2c5da-5dcc-40c2-83bb-728c43e6508d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:49.913707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2c5da-5dcc-40c2-83bb-728c43e6508d None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:49.917595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 152/297] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:49.924489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4779d-812d-447c-8965-64caeea1700d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:49.925487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4779d-812d-447c-8965-64caeea1700d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:49.925487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4779d-812d-447c-8965-64caeea1700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:49.960179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-abb4779d-812d-447c-8965-64caeea1700d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:49.962880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-abb4779d-812d-447c-8965-64caeea1700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:50.075917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 153/298] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:50.077778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e753ec-5c9c-4eb9-8443-f0a87dc1986d None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.078100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e753ec-5c9c-4eb9-8443-f0a87dc1986d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.078350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e753ec-5c9c-4eb9-8443-f0a87dc1986d None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.079981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 154/299] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:50.085694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e7b48-37fb-43a4-854a-8e1bfafbd6f5 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.086476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e7b48-37fb-43a4-854a-8e1bfafbd6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.087041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e7b48-37fb-43a4-854a-8e1bfafbd6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.101772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b30a5-197d-4576-b66b-626af520d61e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:50.106952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fe2e7b48-37fb-43a4-854a-8e1bfafbd6f5 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:50.108584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fe2e7b48-37fb-43a4-854a-8e1bfafbd6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:50.132597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:50.158077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:50.158890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.159170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.159468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.160693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:50.162217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:50.169298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 146/300] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:50.169298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:50.175434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:50.257049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 155/301] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:50.257049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e7b48-37fb-43a4-854a-8e1bfafbd6f5 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:50.328450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa64d4-16be-49b7-b29c-96d70d77cbe9 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:50.394086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:50.395384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.395885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.396373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.417860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:50.425734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e7b48-37fb-43a4-854a-8e1bfafbd6f5 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:50.434715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:50.436772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:50.438623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 147/302] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:50.441999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:50.498427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:50.499139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.499400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.499678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] PATH_INFO: `/v3/groups` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.500379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:50.501888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:50.523927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 156/303] 10.222.0.43 () {60 vars in 1134 bytes} [Wed Jun 29 07:48:50 2022] POST /identity/v3/groups => generated 229 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:50.524493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:50.541305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:50.565433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:50.639063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f9c56-789b-4ecd-a913-b4a6a544c174 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:50.683628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:50.684398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.684646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.684926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.698780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:50.699825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320062c5-1049-4e3d-bfb1-da223ccab920 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:50.714535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:50.715483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:50.716276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:50.717404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 148/304] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:50.726699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:50.749667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:50.751682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.752157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.752632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.753660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:50.754117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:50.756450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:50.769883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 157/305] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:50.778095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:50.900335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadae0c-6dd1-488e-93f0-aa3c5588d350 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:50.905964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:50.924270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:50.948132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d09ac-9da2-4b79-8ff9-e38fd8a27eee None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:50.952680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b89e0-9824-4140-948e-588973d5ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:50.953734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b89e0-9824-4140-948e-588973d5ed8d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.954163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b89e0-9824-4140-948e-588973d5ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.954582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b89e0-9824-4140-948e-588973d5ed8d None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.955776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b89e0-9824-4140-948e-588973d5ed8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:50.956261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b89e0-9824-4140-948e-588973d5ed8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:50.958517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b89e0-9824-4140-948e-588973d5ed8d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:50.977132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 149/306] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:50.977132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c611d-fca2-4a8b-84e2-e7eba9e61f0e None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.977132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c611d-fca2-4a8b-84e2-e7eba9e61f0e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.977132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c611d-fca2-4a8b-84e2-e7eba9e61f0e None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.977132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 150/307] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:50.985298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de940992-68d1-4b7e-b38f-c0469ba4d817 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.985735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de940992-68d1-4b7e-b38f-c0469ba4d817 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.986296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de940992-68d1-4b7e-b38f-c0469ba4d817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.997190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:50.998098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:50.998387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:50.998665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] PATH_INFO: `/v3/users/6ae0ba17410f40e890fe7b99e2fb46cc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:50.999932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] RBAC: Authorizing `identity:get_user(user_id=6ae0ba17410f40e890fe7b99e2fb46cc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:51.002237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:51.004182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 158/308] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:50 2022] GET /identity/v3/users/6ae0ba17410f40e890fe7b99e2fb46cc => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:51.004750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:51.007128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-de940992-68d1-4b7e-b38f-c0469ba4d817 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:51.008378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-de940992-68d1-4b7e-b38f-c0469ba4d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:51.101219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 151/309] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:51.101219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de940992-68d1-4b7e-b38f-c0469ba4d817 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:51.138379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229548-6c56-45f8-8a40-0c0f82bc73b2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:51.189334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:51.190553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:51.191021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:51.191458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] PATH_INFO: `/v3/users/35c7497839d2466e9a92d754971fe348` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:51.216063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] RBAC: Authorizing `identity:get_user(user_id=35c7497839d2466e9a92d754971fe348)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:51.218046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:51.220293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 159/310] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:51 2022] GET /identity/v3/users/35c7497839d2466e9a92d754971fe348 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:51.221404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:51.238757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:51.245399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:51.249576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de940992-68d1-4b7e-b38f-c0469ba4d817 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:51.312482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:51.313754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:51.314199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:51.314682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] PATH_INFO: `/v3/system/users/6ae0ba17410f40e890fe7b99e2fb46cc/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:51.316812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ae0ba17410f40e890fe7b99e2fb46cc, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:51.318630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:51.332372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 152/311] 10.222.0.43 () {58 vars in 1334 bytes} [Wed Jun 29 07:48:51 2022] PUT /identity/v3/system/users/6ae0ba17410f40e890fe7b99e2fb46cc/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:48:51.337040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:51.412696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc8e1d-a542-4dba-b2ea-36db7ce58976 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:51.496698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:51.498723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:51.499472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:51.500336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:51.523234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:51.537968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:51.540618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:51.543006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:51.545460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3425b-f010-4f0b-9a07-95267d66fdad None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:51.545657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 160/312] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:51.549882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:51.605094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:51.606493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:51.606977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:51.607508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:51.609704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] RBAC: Adding query filter params (role.id=00ac4a79282e459f8e0aa6c551f0322d, user.id=35c7497839d2466e9a92d754971fe348, scope.system=all) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:51.610298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:51.614082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:51.624591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 153/313] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jun 29 07:48:51 2022] GET /identity/v3/role_assignments?user.id=35c7497839d2466e9a92d754971fe348&scope.system=all&role.id=00ac4a79282e459f8e0aa6c551f0322d => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:51.690176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:51.740921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a6cd0-baa7-41cd-aca4-bcdfb46c2a78 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:51.809683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:51.810943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:51.814907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:51.815447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:51.817198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:51.817733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:51.820222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:51.840716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 161/314] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:51.850183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:51.882894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:51.969073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749541-832d-4bfc-a042-2268186dc5bb None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:52.006308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:52.007205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:52.007486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:52.007761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] PATH_INFO: `/v3/users/b7da97f4918b4e518965c2d73518969d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:52.009923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] RBAC: Authorizing `identity:get_user(user_id=b7da97f4918b4e518965c2d73518969d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:52.012216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:52.014407 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 154/315] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:51 2022] GET /identity/v3/users/b7da97f4918b4e518965c2d73518969d => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:52.019917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:52.182539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafd5a9-9974-466f-a8ff-728389ca580c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:52.250835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:52.252028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:52.252538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:52.253082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:52.261381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:52.394783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:52.397622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:52.596871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 162/316] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:48:52.603260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:52.738135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:52.748700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd803e-39bd-4f6a-a199-a563a9ead557 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:52.786224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a0d5f-0429-45f9-82c5-161d4dbac51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:52.787412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a0d5f-0429-45f9-82c5-161d4dbac51f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:52.787869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a0d5f-0429-45f9-82c5-161d4dbac51f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:52.788332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a0d5f-0429-45f9-82c5-161d4dbac51f None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:52.790414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a0d5f-0429-45f9-82c5-161d4dbac51f None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:52.792313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a0d5f-0429-45f9-82c5-161d4dbac51f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:52.794676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 155/317] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:52 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:52.801753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:52.869183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:53.001637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8fba2-b5b2-48dd-a092-49252beae2e7 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:53.066790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc217c-05f1-43a3-994a-023f3f0a4027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:53.067998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc217c-05f1-43a3-994a-023f3f0a4027 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:53.068403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc217c-05f1-43a3-994a-023f3f0a4027 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:53.068901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc217c-05f1-43a3-994a-023f3f0a4027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:53.070551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc217c-05f1-43a3-994a-023f3f0a4027 None admin] RBAC: Adding query filter params (role.id=00ac4a79282e459f8e0aa6c551f0322d, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba, user.id=b7da97f4918b4e518965c2d73518969d) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:53.071734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc217c-05f1-43a3-994a-023f3f0a4027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:53.074504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc217c-05f1-43a3-994a-023f3f0a4027 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:53.086525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 163/318] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:48:52 2022] GET /identity/v3/role_assignments?user.id=b7da97f4918b4e518965c2d73518969d&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=00ac4a79282e459f8e0aa6c551f0322d => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:53.385007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbcf8f-757c-4cd2-8bde-b9fca4740eb0 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:53.385554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbcf8f-757c-4cd2-8bde-b9fca4740eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:53.386063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbcf8f-757c-4cd2-8bde-b9fca4740eb0 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:53.388414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 156/319] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:53.394443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d1482-e4de-4da6-8cf4-354fb018c9cc None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:53.394727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d1482-e4de-4da6-8cf4-354fb018c9cc None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:53.395232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d1482-e4de-4da6-8cf4-354fb018c9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:53.415690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e80d1482-e4de-4da6-8cf4-354fb018c9cc None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:53.420429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e80d1482-e4de-4da6-8cf4-354fb018c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:53.552802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 164/320] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:53.595403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbcf8f-757c-4cd2-8bde-b9fca4740eb0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:53.787214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbcf8f-757c-4cd2-8bde-b9fca4740eb0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:53.856883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c6344-b3b6-4fcf-9089-34c909759aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:53.858323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c6344-b3b6-4fcf-9089-34c909759aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:53.858817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c6344-b3b6-4fcf-9089-34c909759aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:53.859296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c6344-b3b6-4fcf-9089-34c909759aa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:53.861800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c6344-b3b6-4fcf-9089-34c909759aa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:53.864615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c6344-b3b6-4fcf-9089-34c909759aa9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:53.868511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 157/321] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:48:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:53.877665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d1482-e4de-4da6-8cf4-354fb018c9cc None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:53.936764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:53.991553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c09f6b-8384-4a67-9102-73d16548bfb6 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:53.991854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c09f6b-8384-4a67-9102-73d16548bfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:53.992219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c09f6b-8384-4a67-9102-73d16548bfb6 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:53.994442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 158/322] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:54.000373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aa4df-e763-4853-85ca-4db694880708 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.000673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aa4df-e763-4853-85ca-4db694880708 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.000956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aa4df-e763-4853-85ca-4db694880708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.021977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ac9aa4df-e763-4853-85ca-4db694880708 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:54.023910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ac9aa4df-e763-4853-85ca-4db694880708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:54.059560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:54.060068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d1482-e4de-4da6-8cf4-354fb018c9cc None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:54.112703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 159/323] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:54.114108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364c9d9-f37f-4799-b276-54b3144b7226 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.114562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364c9d9-f37f-4799-b276-54b3144b7226 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.115014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364c9d9-f37f-4799-b276-54b3144b7226 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.116310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:54.116830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 160/324] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:54.118323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.118630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.118911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] PATH_INFO: `/v3/groups` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.119629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:54.121153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:54.123299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a067-2ec7-47d4-8e55-1c6de7b1eace None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.123588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a067-2ec7-47d4-8e55-1c6de7b1eace None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.124019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a067-2ec7-47d4-8e55-1c6de7b1eace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.138756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 165/325] 10.222.0.43 () {60 vars in 1134 bytes} [Wed Jun 29 07:48:53 2022] POST /identity/v3/groups => generated 226 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:48:54.144452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ee46a067-2ec7-47d4-8e55-1c6de7b1eace None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:54.146044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ee46a067-2ec7-47d4-8e55-1c6de7b1eace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:54.152216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:54.277281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 161/326] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:54.314263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46a067-2ec7-47d4-8e55-1c6de7b1eace None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:54.327104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:54.344033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a38e36-62fc-41d7-8f11-0dd528c484fb None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:54.395096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:54.396403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.397039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.397532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.415992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:54.434779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:54.436365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:54.438659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 166/327] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:54.443286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:54.476107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:54.495933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46a067-2ec7-47d4-8e55-1c6de7b1eace None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:54.544933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:54.546148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.546573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.547057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.569987 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:54.586314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:54.588165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:54.589089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 162/328] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:54.597383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:54.607841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6cee8-be3e-45e6-8752-d707bafc7c61 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:54.651526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:54.652494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.652922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.653390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.654301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:54.654769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:54.656344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:54.667624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 167/329] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:54.673837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:54.692867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:54.787588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96c744-47f6-42f7-942c-72a9f6db6a5b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:54.807855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fbaa7-81d6-4c15-8363-20d372bd9a23 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:54.847670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:54.848864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.849382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.849846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] PATH_INFO: `/v3/users/35c7497839d2466e9a92d754971fe348` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.851328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] RBAC: Authorizing `identity:get_user(user_id=35c7497839d2466e9a92d754971fe348)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:54.853315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:54.855805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 168/330] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:54 2022] GET /identity/v3/users/35c7497839d2466e9a92d754971fe348 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:54.861668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:54.865221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:54.866921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:54.867269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:54.868381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:54.870018 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] RBAC: Adding query filter params (name=member) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:54.870733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:54.873525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:54.893094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 163/331] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:54.903113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:54.924617 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:54.967992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:54.992533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238c12-575b-4ad6-92be-f803c6a564c1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:55.036425 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:55.037607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:55.038075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:55.041587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] PATH_INFO: `/v3/system/users/35c7497839d2466e9a92d754971fe348/roles/00ac4a79282e459f8e0aa6c551f0322d` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:55.043658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=35c7497839d2466e9a92d754971fe348, role_id=00ac4a79282e459f8e0aa6c551f0322d)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:55.045341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:55.051702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02c2ad-eac4-43d9-bf3d-ee94657c670e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:55.066129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 169/332] 10.222.0.43 () {58 vars in 1334 bytes} [Wed Jun 29 07:48:54 2022] PUT /identity/v3/system/users/35c7497839d2466e9a92d754971fe348/roles/00ac4a79282e459f8e0aa6c551f0322d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:48:55.107110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124854e-d9e3-4797-9de4-6ddf9a383bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:55.107946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124854e-d9e3-4797-9de4-6ddf9a383bdd None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:55.108251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124854e-d9e3-4797-9de4-6ddf9a383bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:55.108638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124854e-d9e3-4797-9de4-6ddf9a383bdd None admin] PATH_INFO: `/v3/users/6ae0ba17410f40e890fe7b99e2fb46cc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:55.114075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124854e-d9e3-4797-9de4-6ddf9a383bdd None admin] RBAC: Authorizing `identity:get_user(user_id=6ae0ba17410f40e890fe7b99e2fb46cc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:55.115776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124854e-d9e3-4797-9de4-6ddf9a383bdd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:55.118755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 164/333] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:48:54 2022] GET /identity/v3/users/6ae0ba17410f40e890fe7b99e2fb46cc => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:55.125384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:55.368798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fc44d-51d9-40be-8158-070cf824fa2f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:55.443383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a2a81-ca71-4f03-a31c-f79d3a4c051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:55.444909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a2a81-ca71-4f03-a31c-f79d3a4c051a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:55.445699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a2a81-ca71-4f03-a31c-f79d3a4c051a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:55.447087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a2a81-ca71-4f03-a31c-f79d3a4c051a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:55.449246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a2a81-ca71-4f03-a31c-f79d3a4c051a None admin] RBAC: Adding query filter params (role.id=acffa214db594ed9803f8f5376b15d42, user.id=6ae0ba17410f40e890fe7b99e2fb46cc, scope.system=all) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:55.450133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a2a81-ca71-4f03-a31c-f79d3a4c051a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:55.453084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a2a81-ca71-4f03-a31c-f79d3a4c051a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:55.480641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 170/334] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jun 29 07:48:55 2022] GET /identity/v3/role_assignments?user.id=6ae0ba17410f40e890fe7b99e2fb46cc&scope.system=all&role.id=acffa214db594ed9803f8f5376b15d42 => generated 510 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:56.858649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c0193-cd09-45ae-91ff-c384a4593e59 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:56.859163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c0193-cd09-45ae-91ff-c384a4593e59 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:56.859237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c0193-cd09-45ae-91ff-c384a4593e59 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:56.861014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 165/335] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:56.867545 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be9223-039d-4502-9372-656d0219fd8d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:56.868581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be9223-039d-4502-9372-656d0219fd8d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:56.868686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be9223-039d-4502-9372-656d0219fd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:56.874156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:56.892357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-a2be9223-039d-4502-9372-656d0219fd8d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:56.893599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a2be9223-039d-4502-9372-656d0219fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:56.984400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 171/336] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:48:57.029520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c0193-cd09-45ae-91ff-c384a4593e59 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:57.164392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aab4b-88de-44e9-8d85-417c324121b6 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:57.164795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aab4b-88de-44e9-8d85-417c324121b6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:57.164998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aab4b-88de-44e9-8d85-417c324121b6 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:57.166761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 172/337] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:57.173389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d44e9-60ad-4119-b2ae-980c2f428fca None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:57.174309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d44e9-60ad-4119-b2ae-980c2f428fca None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:57.174816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d44e9-60ad-4119-b2ae-980c2f428fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:57.200065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:57.213374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-754d44e9-60ad-4119-b2ae-980c2f428fca None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:57.214980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-754d44e9-60ad-4119-b2ae-980c2f428fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:57.255544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c0193-cd09-45ae-91ff-c384a4593e59 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:57.349025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:57.350276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:57.350764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:57.351257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:57.382970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:57.397980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:57.398984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:48:57.399822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 166/338] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:48:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:48:57.407239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 173/339] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:48:57.408098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:57.447669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d44e9-60ad-4119-b2ae-980c2f428fca None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:57.637252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:57.642201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b11c5-2790-492a-abb7-125fb673b0ec None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:57.711459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:57.714868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:57.714868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:57.714868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:57.715297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:57.715435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:57.716757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:57.726378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:57.733820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d44e9-60ad-4119-b2ae-980c2f428fca None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:57.734757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 167/340] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:48:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:57.741611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:57.819865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:57.821082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:57.821516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:57.821925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:57.857999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:48:57.866213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:48:57.867271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:48:57.873316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:48:57.876156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 174/341] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:48:57.882186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:58.025810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c712e0-ba1e-4ded-af3b-94ab061695b8 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:58.110833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:58.111766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:58.112244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:58.112691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:58.230177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03093dd-b0d6-4794-b0d1-b9b505e7a3f1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:58.281197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:58.286637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:58.289129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:58.308801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:58.363147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:58.364329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:58.364850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:58.365525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:58.366875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:58.367400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:58.370873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:58.405179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 175/342] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:48:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 518 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:58.412655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:58.546973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 168/343] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 807 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:48:58.562297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:58.694553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06dac7d-87dc-41ca-bcd9-b7abb507e6c7 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:58.716736 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014081-d50e-42f2-b6b0-3af679fd9bca None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:58.719639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:58.761824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:58.762519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:58.762810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:58.763095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:58.793815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:58.794720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:58.795007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:58.795360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:58.796645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:58.798524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:58.800568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 169/344] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:58 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:58.809172 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:58.858191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:58.880183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:58.881731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:58.976604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e67761-b4f2-4647-9ce3-fe2f5087f9e9 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:59.033026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:59.033999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.034348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.034583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.045766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:59.047275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:59.049443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 170/345] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:48:58 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:59.059902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:59.070095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 176/346] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:48:59.095285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:59.114981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:59.311882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:59.337671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d02147-0f02-438d-8a04-ba3fd80a0cd1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:59.339613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2f569-f645-4c59-8da0-b0c6f0b4cf45 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:59.389840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:59.390734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.391039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.391297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.392585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:59.394136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:59.395962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 177/347] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:48:59 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:59.403163 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:59.434861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:59.435562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.435845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.436123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.437167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:59.437363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:59.438813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:59.450043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 171/348] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:48:59 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 260 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:48:59.524179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f788bf-4c47-484f-b1ed-9cd5cd88e3aa None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:59.566276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ee9a5-9c01-40d4-94d3-e26d16337e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:59.567492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ee9a5-9c01-40d4-94d3-e26d16337e83 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.567982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ee9a5-9c01-40d4-94d3-e26d16337e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.568438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ee9a5-9c01-40d4-94d3-e26d16337e83 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.570051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ee9a5-9c01-40d4-94d3-e26d16337e83 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:59.572578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ee9a5-9c01-40d4-94d3-e26d16337e83 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:59.575000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 178/349] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:48:59 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:59.586204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:48:59.741651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:48:59.751681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfbda2-e4a0-4c1a-be22-4ad14a6097a4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:48:59.818011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767e6df-5c28-469d-a16b-b9b2837d243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:48:59.818633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767e6df-5c28-469d-a16b-b9b2837d243e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.818925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767e6df-5c28-469d-a16b-b9b2837d243e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.819212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767e6df-5c28-469d-a16b-b9b2837d243e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.820424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767e6df-5c28-469d-a16b-b9b2837d243e None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:48:59.821473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767e6df-5c28-469d-a16b-b9b2837d243e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:48:59.823826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767e6df-5c28-469d-a16b-b9b2837d243e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:48:59.832316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 172/350] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:48:59 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=acffa214db594ed9803f8f5376b15d42 => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:48:59.865578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3a1bf-6a75-4797-9761-e143bef7f154 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.866211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3a1bf-6a75-4797-9761-e143bef7f154 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.866688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3a1bf-6a75-4797-9761-e143bef7f154 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.868906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 179/351] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:48:59.882988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64634802-196c-4375-891e-a37e22c9011a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.883465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64634802-196c-4375-891e-a37e22c9011a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.883873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64634802-196c-4375-891e-a37e22c9011a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.906351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-64634802-196c-4375-891e-a37e22c9011a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:48:59.907611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-64634802-196c-4375-891e-a37e22c9011a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:48:59.950994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788740-e6b3-41d7-a9a8-a7d7c33ae1c7 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.951515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788740-e6b3-41d7-a9a8-a7d7c33ae1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.951944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788740-e6b3-41d7-a9a8-a7d7c33ae1c7 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:48:59.954972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 180/352] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:48:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:48:59.966449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f0dcb-28a9-46da-816c-4995833e3a68 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:48:59.966936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f0dcb-28a9-46da-816c-4995833e3a68 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:48:59.967405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f0dcb-28a9-46da-816c-4995833e3a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:00.019786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 173/353] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:00.023272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e17f0dcb-28a9-46da-816c-4995833e3a68 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:00.025942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e17f0dcb-28a9-46da-816c-4995833e3a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:00.051785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64634802-196c-4375-891e-a37e22c9011a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:00.066159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:00.132442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 181/354] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:00.186609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f0dcb-28a9-46da-816c-4995833e3a68 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:00.265399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:00.301161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64634802-196c-4375-891e-a37e22c9011a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:00.410578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:00.411338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:00.411628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:00.411909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:00.436567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:00.449080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:00.450074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 07:49:00.450885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:00.453430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 174/355] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:00.456351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:00.513275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f0dcb-28a9-46da-816c-4995833e3a68 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:00.589100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:00.590316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:00.590798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:00.591275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:00.624212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:00.634036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:00.634991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:00.635782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:00.638733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 182/356] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:00.646823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:00.673654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:00.755684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473735f-2674-411f-8f75-b026c1927549 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:00.833762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:00.834903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:00.835400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:00.835886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:00.837881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:00.838406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:00.845012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:00.857224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 175/357] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:49:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:00.866026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:00.902140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:00.910464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200ec50-88fe-403b-af78-43b43d61cf50 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:00.991765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac086d6-50af-49aa-a671-eba67bf0f549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:00.993952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac086d6-50af-49aa-a671-eba67bf0f549 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:00.994500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac086d6-50af-49aa-a671-eba67bf0f549 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:00.994980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac086d6-50af-49aa-a671-eba67bf0f549 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:00.996136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac086d6-50af-49aa-a671-eba67bf0f549 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:00.996705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac086d6-50af-49aa-a671-eba67bf0f549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:00.998358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac086d6-50af-49aa-a671-eba67bf0f549 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:01.008833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 183/358] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:49:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:01.010601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7a882-07a3-41ce-9327-63d64a335f34 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.011138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7a882-07a3-41ce-9327-63d64a335f34 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.011636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7a882-07a3-41ce-9327-63d64a335f34 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.013264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 184/359] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:01.014658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61071e-a1f8-495f-82a5-dbe3e3d19096 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.015158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61071e-a1f8-495f-82a5-dbe3e3d19096 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.015631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61071e-a1f8-495f-82a5-dbe3e3d19096 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.017464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 185/360] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:01.018460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61071e-a1f8-495f-82a5-dbe3e3d19096 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:01.084449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68fd8c-cf1d-430d-a173-eee95fa1cd9e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:01.112223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:01.151828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61071e-a1f8-495f-82a5-dbe3e3d19096 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:01.152416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e2124-2e9f-422e-a0fb-291a2687662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:01.153920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e2124-2e9f-422e-a0fb-291a2687662a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.154579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e2124-2e9f-422e-a0fb-291a2687662a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.155038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e2124-2e9f-422e-a0fb-291a2687662a None admin] PATH_INFO: `/v3/users/35c7497839d2466e9a92d754971fe348` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.157663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e2124-2e9f-422e-a0fb-291a2687662a None admin] RBAC: Authorizing `identity:get_user(user_id=35c7497839d2466e9a92d754971fe348)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:01.160238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e2124-2e9f-422e-a0fb-291a2687662a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:01.169350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 176/361] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:49:00 2022] GET /identity/v3/users/35c7497839d2466e9a92d754971fe348 => generated 293 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:01.172052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98493c1e-24fc-482b-b949-8d52e5fde1a3 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.172840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98493c1e-24fc-482b-b949-8d52e5fde1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.173497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98493c1e-24fc-482b-b949-8d52e5fde1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.198747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:01.199855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.200339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.200833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.210773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-98493c1e-24fc-482b-b949-8d52e5fde1a3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:01.213220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-98493c1e-24fc-482b-b949-8d52e5fde1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:01.283152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:01.285164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:01.350191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 177/362] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:01.352796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54527061-010a-47c4-8ce8-f84eab2bfa79 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.353340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54527061-010a-47c4-8ce8-f84eab2bfa79 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.353834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54527061-010a-47c4-8ce8-f84eab2bfa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.385323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-54527061-010a-47c4-8ce8-f84eab2bfa79 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:01.386690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:01.389617 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-54527061-010a-47c4-8ce8-f84eab2bfa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:01.395163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:01.429462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 186/363] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:01.431224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:01.523399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 178/364] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:01.525073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54527061-010a-47c4-8ce8-f84eab2bfa79 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:01.608158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403ef13-b5c6-4731-a339-2d690f108517 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:01.687055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54527061-010a-47c4-8ce8-f84eab2bfa79 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:01.701230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:01.702857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.703794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.704705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.706801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] RBAC: Adding query filter params (role.id=00ac4a79282e459f8e0aa6c551f0322d, user.id=35c7497839d2466e9a92d754971fe348, scope.system=all) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:01.707891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:01.711082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:01.731616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:01.732464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:01.732756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:01.734365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:01.743202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 187/365] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jun 29 07:49:01 2022] GET /identity/v3/role_assignments?user.id=35c7497839d2466e9a92d754971fe348&scope.system=all&role.id=00ac4a79282e459f8e0aa6c551f0322d => generated 510 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:01.745847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:01.751299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:01.763848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:01.780412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:01.781454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:01.782253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:01.785232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 179/366] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:01.786626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:01.836059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:02.038744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403852aa-e58b-4bcc-ba24-fc32fe5d47ec None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:02.038928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be992cb0-15b8-4dbe-919d-afe9203b34b5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:02.113214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:02.119445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:02.119445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:02.119445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:02.121696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:02.123246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:02.123763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:02.124278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:02.126620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:02.130020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:02.135184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 188/367] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:01 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:02.139957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:02.148095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:02.167061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:02.168865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:02.174047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:02.178378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 180/368] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:49:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:02.180661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:02.311885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:02.372114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf73a-a609-43a3-baa7-e94d6834083d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:02.386625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:02.401016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3753d0c-129e-41fb-bca6-65c658173293 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:02.465864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:02.466823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:02.467170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:02.467502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:02.469277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:02.469594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:02.471603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:02.494748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:02.496374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:02.497187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:02.497947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:02.500413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:02.503286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:02.506683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 189/369] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:02.508018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:02.509107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 181/370] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:02 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:02.514905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:02.751168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb8941-19fd-431d-be84-3891a14829ba None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:02.759436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd374c86-0d3a-4289-9f31-452af84f3f7f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:02.816767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:02.835453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c5f6-798b-4f9f-99d2-53379d814d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:02.838099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c5f6-798b-4f9f-99d2-53379d814d3f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:02.839009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c5f6-798b-4f9f-99d2-53379d814d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:02.839920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c5f6-798b-4f9f-99d2-53379d814d3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:02.842246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:02.843392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:02.843889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:02.844408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:02.845917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:02.846489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:02.851207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:02.869117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 190/371] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:02.870094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:02.925231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:02.997153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02c5f6-798b-4f9f-99d2-53379d814d3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:02.999758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02c5f6-798b-4f9f-99d2-53379d814d3f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:03.025762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072e8af-fe8e-4998-a333-337d4626da87 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:03.087411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:03.088916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:03.089868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:03.090352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:03.091732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:03.092261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:03.094223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:03.103554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 191/372] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:02 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=acffa214db594ed9803f8f5376b15d42 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:03.109005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:03.225868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 182/373] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:03.228873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42098b-9bbd-46e2-81b0-4121b758f428 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:03.236773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42098b-9bbd-46e2-81b0-4121b758f428 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:03.237373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42098b-9bbd-46e2-81b0-4121b758f428 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:03.240543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 183/374] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:03.242077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42098b-9bbd-46e2-81b0-4121b758f428 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:03.306305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:03.330715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:03.344462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c76a5d-4ca4-48f1-a9ab-0807980d79b6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:03.387852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb48d61-8160-4c67-9326-5dd7275c5d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:03.388526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb48d61-8160-4c67-9326-5dd7275c5d3e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:03.388820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb48d61-8160-4c67-9326-5dd7275c5d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:03.389146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb48d61-8160-4c67-9326-5dd7275c5d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:03.503010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42098b-9bbd-46e2-81b0-4121b758f428 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:03.530323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb48d61-8160-4c67-9326-5dd7275c5d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:03.531883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb48d61-8160-4c67-9326-5dd7275c5d3e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:03.588075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8e398-eee7-466a-97da-5bf10e8236a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:03.589953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e398-eee7-466a-97da-5bf10e8236a7 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:03.590754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e398-eee7-466a-97da-5bf10e8236a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:03.591193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e398-eee7-466a-97da-5bf10e8236a7 None admin] PATH_INFO: `/v3/projects/c4e7f90bb9da44679fd899e1c8b90999` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:03.594428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8e398-eee7-466a-97da-5bf10e8236a7 None admin] RBAC: Authorizing `identity:get_project(project_id=c4e7f90bb9da44679fd899e1c8b90999)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:03.598801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8e398-eee7-466a-97da-5bf10e8236a7 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:03.603422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 184/375] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:03 2022] GET /identity/v3/projects/c4e7f90bb9da44679fd899e1c8b90999 => generated 344 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:03.606448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e331e8-d9f4-4d06-b8ce-9f203edbda6e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:03.606971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e331e8-d9f4-4d06-b8ce-9f203edbda6e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:03.607488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e331e8-d9f4-4d06-b8ce-9f203edbda6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:03.638841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-06e331e8-d9f4-4d06-b8ce-9f203edbda6e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:03.640113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-06e331e8-d9f4-4d06-b8ce-9f203edbda6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:03.723816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:03.759935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 192/376] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:03.761189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b8546-59b1-40f2-8544-3cde86f462dd None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:03.761477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b8546-59b1-40f2-8544-3cde86f462dd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:03.761755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b8546-59b1-40f2-8544-3cde86f462dd None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:03.766753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 193/377] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:03.766753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b8546-59b1-40f2-8544-3cde86f462dd None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:03.780840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 185/378] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:03.782975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e331e8-d9f4-4d06-b8ce-9f203edbda6e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:03.842078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:03.926952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b8546-59b1-40f2-8544-3cde86f462dd None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:03.975223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e331e8-d9f4-4d06-b8ce-9f203edbda6e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:04.007064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a92b7-10f2-4c8a-8203-ccc1976cc18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.009392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a92b7-10f2-4c8a-8203-ccc1976cc18a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.009812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a92b7-10f2-4c8a-8203-ccc1976cc18a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.010362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a92b7-10f2-4c8a-8203-ccc1976cc18a None admin] PATH_INFO: `/v3/groups/2d41a6dee10c49a49b9dd559b081c7c0` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.023548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a92b7-10f2-4c8a-8203-ccc1976cc18a None admin] RBAC: Authorizing `identity:get_group(group_id=2d41a6dee10c49a49b9dd559b081c7c0)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:04.027349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a92b7-10f2-4c8a-8203-ccc1976cc18a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:04.032841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 194/379] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:03 2022] GET /identity/v3/groups/2d41a6dee10c49a49b9dd559b081c7c0 => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:04.035472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e7501-ddac-48e3-b686-c97924c14004 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.036222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e7501-ddac-48e3-b686-c97924c14004 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.036874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e7501-ddac-48e3-b686-c97924c14004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.041563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.042587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.042977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.043361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.044831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:04.046444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:04.048834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 186/380] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:03 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:04.049693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:04.068943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-616e7501-ddac-48e3-b686-c97924c14004 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:04.071925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-616e7501-ddac-48e3-b686-c97924c14004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:04.175236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:04.219981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 195/381] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:04.221586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e7501-ddac-48e3-b686-c97924c14004 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:04.234545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aeefe-2818-46c0-b3b2-32920304becf None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:04.236725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:04.309520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.309520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.309520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.309520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.332287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:04.347628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:04.349590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:04.351206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 187/382] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:49:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:04.353787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:04.419964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e7501-ddac-48e3-b686-c97924c14004 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:04.482880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.483276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.483772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.486467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.486467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] RBAC: Adding query filter params (group.id=2d41a6dee10c49a49b9dd559b081c7c0, role.id=fe0d00714612495091430762d114121a, scope.project.id=c4e7f90bb9da44679fd899e1c8b90999) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:04.486467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:04.489663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:04.501809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 196/383] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:04 2022] GET /identity/v3/role_assignments?group.id=2d41a6dee10c49a49b9dd559b081c7c0&scope.project.id=c4e7f90bb9da44679fd899e1c8b90999&role.id=fe0d00714612495091430762d114121a => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:04.503073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:04.560560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423007e-0d35-44bd-a0cd-74acff60799b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:04.577773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:04.639382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.640844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.641427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.641901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.644072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:04.646816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:04.651861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c278-941a-436c-ae7f-dacd87d4d3cd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:04.657561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 188/384] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:04 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:04.658962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:04.701793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:04.727445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.728929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.729494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.730163 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.747104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:04.755963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:04.756921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:04.757755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:04.766178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 197/385] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:04.767562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:04.820087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011292b-986c-4d73-a02d-04540c2b737c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:04.881254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.881949 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.882277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.882556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.883862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:04.884321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:04.886219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:04.900595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 189/386] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:04.901676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:04.916461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab6955-1efe-4a05-b791-666a09f473b5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:04.921094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:04.985008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:04.986184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:04.986692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:04.987204 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:04.988875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:04.989467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:04.992314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:05.001399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:05.003938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 198/387] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:04 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:05.005488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:05.056641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e3c7d-319e-4b0b-975d-dab2700ce21a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:05.175581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:05.177774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:05.178351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:05.178867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:05.181272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] RBAC: Adding query filter params (name=member) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:05.182038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:05.185278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:05.207140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 190/388] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:49:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:05.214684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:05.297311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bebb87-92e7-4423-a413-d7fc1d1d31dc None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:05.330111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8852e8-64c6-4929-bb0e-b8e458cfe7db None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:05.379595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:05.380470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:05.380756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:05.381069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:05.382519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:05.384498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:05.387260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 199/389] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:05 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:05.393604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:05.399249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:05.400538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:05.400830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:05.401135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:05.403300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:05.404769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:05.412035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 191/390] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:05 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:05.421836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:05.465571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:05.534382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:05.765344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862222d-ffaf-4535-b69b-a0c5804aa925 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:05.778573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd323bd5-7300-4575-947c-0ac2a707e48b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:05.871538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:05.874335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:05.875870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:05.877348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:05.881222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:05.882550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:05.886792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:05.887603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:05.889532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:05.890374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:05.895003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:05.898064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:05.900432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 200/391] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:05 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 507 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:05.904633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 192/392] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:05 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:05.910725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:05.912630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:06.173623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:06.213811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5e2dd-9311-40ef-8220-d671c612d9e9 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:06.265834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd399b5d-0d3f-450d-a5df-cbe97cd0ea1f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:06.270630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:06.274564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae0bff-2ecc-4afd-b2b9-2ab83d1e55af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:06.275601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae0bff-2ecc-4afd-b2b9-2ab83d1e55af None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:06.275892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae0bff-2ecc-4afd-b2b9-2ab83d1e55af None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:06.276170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae0bff-2ecc-4afd-b2b9-2ab83d1e55af None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:06.278545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae0bff-2ecc-4afd-b2b9-2ab83d1e55af None admin] RBAC: Authorizing `identity:create_grant(project_id=6bf43a1fb0074f3c8f0a85646b793ebc, group_id=6a2574d6b2ce498eac0349172d4f6d12, role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:06.280014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae0bff-2ecc-4afd-b2b9-2ab83d1e55af None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:06.298007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 193/393] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jun 29 07:49:05 2022] PUT /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:06.299576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc1215-415a-4804-aece-304c92cced58 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:06.299872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc1215-415a-4804-aece-304c92cced58 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:06.300203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc1215-415a-4804-aece-304c92cced58 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:06.301697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 194/394] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:06.311621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90716-11b0-4ef8-bf9d-7ebe698712ad None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:06.312059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90716-11b0-4ef8-bf9d-7ebe698712ad None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:06.312443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90716-11b0-4ef8-bf9d-7ebe698712ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:06.332751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-4dd90716-11b0-4ef8-bf9d-7ebe698712ad None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:06.334405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4dd90716-11b0-4ef8-bf9d-7ebe698712ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:06.387932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:06.389743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:06.390257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:06.390759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:06.400698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] RBAC: Authorizing `identity:create_grant(project_id=6bf43a1fb0074f3c8f0a85646b793ebc, group_id=6a2574d6b2ce498eac0349172d4f6d12, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:06.403855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:06.420426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 195/395] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:06.447075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 201/396] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jun 29 07:49:05 2022] PUT /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:06.447474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd90716-11b0-4ef8-bf9d-7ebe698712ad None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:06.584258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:06.635181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd90716-11b0-4ef8-bf9d-7ebe698712ad None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:06.702668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:06.703997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:06.704504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:06.705039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:06.734376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:06.750908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:06.751883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:06.752698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:06.753459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 196/397] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:06.761585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:07.051899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbefcbb-7915-4fb9-aa70-90025455aaab None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:07.135048 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:07.137948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:07.138486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:07.138995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:07.140454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:07.141076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:07.143877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:07.171347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 202/398] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:49:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:07.178274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:07.455208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae1098-fb43-4aff-9974-ecfc1185e0e0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:07.536799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:07.551578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605c42c-d6e7-433b-8ad0-67a36ef2b577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:07.553112 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c42c-d6e7-433b-8ad0-67a36ef2b577 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:07.553631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c42c-d6e7-433b-8ad0-67a36ef2b577 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:07.554140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c42c-d6e7-433b-8ad0-67a36ef2b577 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:07.561055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605c42c-d6e7-433b-8ad0-67a36ef2b577 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:07.563890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605c42c-d6e7-433b-8ad0-67a36ef2b577 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:07.567184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 197/399] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:07 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:07.573860 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:07.599663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3916f-8a9e-42f0-a9af-7ec02fec7cdd None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:07.600182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3916f-8a9e-42f0-a9af-7ec02fec7cdd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:07.600693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3916f-8a9e-42f0-a9af-7ec02fec7cdd None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:07.603859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 198/400] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:07.613200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99218e-cb1c-481c-a052-0fc6609e893a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:07.613700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99218e-cb1c-481c-a052-0fc6609e893a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:07.614310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99218e-cb1c-481c-a052-0fc6609e893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:07.645524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:07.661280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2e99218e-cb1c-481c-a052-0fc6609e893a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:07.663896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2e99218e-cb1c-481c-a052-0fc6609e893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:07.771335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44235f-cdbe-465e-81cd-48d329eb8401 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:07.815259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:07.817174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:07.817464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:07.817743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:07.820311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:07.822545 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:07.824529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 203/401] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:07 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:07.834016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:07.837241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 199/402] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:07.860937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99218e-cb1c-481c-a052-0fc6609e893a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:08.036785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:08.050701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9efa2-8042-438d-8f06-0136ffecb8bd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:08.068081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99218e-cb1c-481c-a052-0fc6609e893a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:08.092966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be298f57-4361-4599-8628-2c418779a265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:08.093929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be298f57-4361-4599-8628-2c418779a265 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.094217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be298f57-4361-4599-8628-2c418779a265 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.094494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be298f57-4361-4599-8628-2c418779a265 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.096763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be298f57-4361-4599-8628-2c418779a265 None admin] RBAC: Authorizing `identity:create_grant(project_id=114512b9b8fe40f4a5d4d34ea099a4ba, group_id=6a2574d6b2ce498eac0349172d4f6d12, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:08.098317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be298f57-4361-4599-8628-2c418779a265 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:08.108295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:08.116166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:08.117881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.118489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.118616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 204/403] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jun 29 07:49:07 2022] PUT /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:49:08.119190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.119999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf93b3-8173-4e07-8b2a-68af3f41dd6e None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.120298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf93b3-8173-4e07-8b2a-68af3f41dd6e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.120575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf93b3-8173-4e07-8b2a-68af3f41dd6e None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.122523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 205/404] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:08.130337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd588-e209-4409-a987-9f4b8dd98c95 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.130707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd588-e209-4409-a987-9f4b8dd98c95 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.131096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd588-e209-4409-a987-9f4b8dd98c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.142904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:08.153287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:08.154299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:08.155124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 200/405] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:08.158728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-934bd588-e209-4409-a987-9f4b8dd98c95 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:08.161150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-934bd588-e209-4409-a987-9f4b8dd98c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:08.161550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:08.307901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 206/406] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:08.347647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd588-e209-4409-a987-9f4b8dd98c95 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:08.379342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f93d-8ed6-47e8-b5be-e2669e235764 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:08.421364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:08.422448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.423188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.423663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.424660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:08.425208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:08.426897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:08.438384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 201/407] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:08.450040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:08.519648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:08.542676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:08.577540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd588-e209-4409-a987-9f4b8dd98c95 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:08.583278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc1c78-9ddd-4d41-a466-019a0a7e751b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:08.624917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:08.626325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.626856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.627567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] PATH_INFO: `/v3/groups/2d41a6dee10c49a49b9dd559b081c7c0` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.629164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] RBAC: Authorizing `identity:get_group(group_id=2d41a6dee10c49a49b9dd559b081c7c0)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:08.631124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:08.634408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 202/408] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:08 2022] GET /identity/v3/groups/2d41a6dee10c49a49b9dd559b081c7c0 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:08.640490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:08.644425 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:08.646060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.646557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.647040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.667803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:08.684457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:08.685578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:08.686400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:08.686995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 207/409] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:08.702909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:08.776485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8b5d-4be2-4cbf-9389-53e87603ba90 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:08.805558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:08.831582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:08.837732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:08.838471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:08.839163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] PATH_INFO: `/v3/projects/c4e7f90bb9da44679fd899e1c8b90999` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:08.842297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] RBAC: Authorizing `identity:get_project(project_id=c4e7f90bb9da44679fd899e1c8b90999)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:08.847814 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:08.861931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 203/410] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:08 2022] GET /identity/v3/projects/c4e7f90bb9da44679fd899e1c8b90999 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:08.869465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:09.079321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb1392-ce55-49a7-9e1c-8776a4e7edb2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:09.175357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036439f-2cea-4742-a9d4-aeae919849b3 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:09.222167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:09.223286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:09.223764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:09.224235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:09.225613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:09.226207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:09.237552 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:09.257711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:09.260302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:09.264160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:09.264936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] PATH_INFO: `/v3/projects/c4e7f90bb9da44679fd899e1c8b90999/groups/2d41a6dee10c49a49b9dd559b081c7c0/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:09.270794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 208/411] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 569 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:09.273571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] RBAC: Authorizing `identity:create_grant(project_id=c4e7f90bb9da44679fd899e1c8b90999, group_id=2d41a6dee10c49a49b9dd559b081c7c0, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:09.276950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:09.278743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:09.304215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:09.314164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 204/412] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jun 29 07:49:08 2022] PUT /identity/v3/projects/c4e7f90bb9da44679fd899e1c8b90999/groups/2d41a6dee10c49a49b9dd559b081c7c0/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:49:09.536416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd2e08-1bb4-47db-a4f0-ffabf494208b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:09.601894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb2c54-54db-417e-9542-0725a25895f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:09.603394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb2c54-54db-417e-9542-0725a25895f6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:09.603900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb2c54-54db-417e-9542-0725a25895f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:09.604403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb2c54-54db-417e-9542-0725a25895f6 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:09.606531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb2c54-54db-417e-9542-0725a25895f6 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:09.609309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb2c54-54db-417e-9542-0725a25895f6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:09.612456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 209/413] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:09 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:09.619643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:09.672490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd8aa-af20-48cd-87f1-56a5b8f74dec None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:09.673271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd8aa-af20-48cd-87f1-56a5b8f74dec None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:09.673769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd8aa-af20-48cd-87f1-56a5b8f74dec None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:09.677214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 210/414] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:09.690519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282abab-1a6d-45cc-9559-e306b29122db None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:09.691153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282abab-1a6d-45cc-9559-e306b29122db None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:09.692208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282abab-1a6d-45cc-9559-e306b29122db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:09.724744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5282abab-1a6d-45cc-9559-e306b29122db None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:09.727284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5282abab-1a6d-45cc-9559-e306b29122db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:09.768287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:09.860427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 211/415] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:09.903054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282abab-1a6d-45cc-9559-e306b29122db None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:09.908228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:09.956467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9db23-48b1-4633-92dc-4ec5c361877f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:10.085463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:10.089526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.090030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.090534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.096291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282abab-1a6d-45cc-9559-e306b29122db None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:10.098033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:10.100774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:10.104676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 205/416] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:09 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:10.121391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:10.212238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88ca2-1c27-4422-8a6e-6af44140fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:10.213130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88ca2-1c27-4422-8a6e-6af44140fd69 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.213423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88ca2-1c27-4422-8a6e-6af44140fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.213701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88ca2-1c27-4422-8a6e-6af44140fd69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.227153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf88ca2-1c27-4422-8a6e-6af44140fd69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-8cf88ca2-1c27-4422-8a6e-6af44140fd69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:10.235409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:10.237353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 212/417] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:10.238943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac85e55-03e7-4c8c-a6e0-3b1527eaed5c None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.240311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac85e55-03e7-4c8c-a6e0-3b1527eaed5c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.240311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac85e55-03e7-4c8c-a6e0-3b1527eaed5c None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.244822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 213/418] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:10.246286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a46986-5f71-4e49-a006-e21d8745a700 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.246578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a46986-5f71-4e49-a006-e21d8745a700 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.246863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a46986-5f71-4e49-a006-e21d8745a700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.266714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-31a46986-5f71-4e49-a006-e21d8745a700 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:10.267378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-31a46986-5f71-4e49-a006-e21d8745a700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:10.315951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b763da-ec7b-4e9b-b6d4-12885b932097 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:10.337892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:10.359927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:10.362629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:10.363776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.364157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.365142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.372108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] RBAC: Authorizing `identity:create_grant(project_id=114512b9b8fe40f4a5d4d34ea099a4ba, group_id=6a2574d6b2ce498eac0349172d4f6d12, role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:10.373952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:10.377701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 214/419] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:10.379002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a46986-5f71-4e49-a006-e21d8745a700 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:10.409133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 206/420] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jun 29 07:49:10 2022] PUT /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba/groups/6a2574d6b2ce498eac0349172d4f6d12/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:10.449104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:10.507563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a46986-5f71-4e49-a006-e21d8745a700 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:10.551210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:10.552311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.552841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.553349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.554328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:10.557015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:10.558958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:10.569418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 215/421] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:49:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:10.576434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:10.653580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:10.709684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e634f-4f26-4e95-a0cb-74c58b00df8a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:10.757232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:10.759132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.759928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.760703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.829663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ac8d-9424-44da-827a-dbf6ffaaf451 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:10.856675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:10.858327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:10.888342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:10.889934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:10.890290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:10.890724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:10.910155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:10.920194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:10.921271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:10.922499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 207/422] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:49:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 474 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:10.929711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:10.938342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:10.992770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:11.029134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 216/423] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:11.034931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:11.119091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924757-b924-44fe-a5df-d4da9bdbb1df None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:11.193796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:11.194629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:11.195112 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:11.195539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:11.196361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:11.196683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:11.199218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:11.212690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 208/424] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:11.221607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:11.255333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dbd77-4d59-4866-acf2-4555000625c1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:11.339451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:11.340345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:11.340630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:11.340906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:11.350067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:11.351637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:11.353894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 217/425] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:11 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:11.359695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:11.412147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:11.413128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:11.417433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf80a2-3df5-470c-9fca-e5bf4b85c78d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:11.497132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bda0d-93a2-49f7-b9b0-dfdb3f1fe352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:11.497132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bda0d-93a2-49f7-b9b0-dfdb3f1fe352 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:11.497132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bda0d-93a2-49f7-b9b0-dfdb3f1fe352 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:11.497132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bda0d-93a2-49f7-b9b0-dfdb3f1fe352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:11.609256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fed1df-f285-42c7-b72d-f6c59eaa7eb9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:11.657222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bda0d-93a2-49f7-b9b0-dfdb3f1fe352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:11.658962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bda0d-93a2-49f7-b9b0-dfdb3f1fe352 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:11.665117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef638c3c-5aaa-42a2-8cd8-7f3c01cf2227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:11.666444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638c3c-5aaa-42a2-8cd8-7f3c01cf2227 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:11.666870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638c3c-5aaa-42a2-8cd8-7f3c01cf2227 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:11.667285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638c3c-5aaa-42a2-8cd8-7f3c01cf2227 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:11.669921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef638c3c-5aaa-42a2-8cd8-7f3c01cf2227 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:11.671441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef638c3c-5aaa-42a2-8cd8-7f3c01cf2227 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:11.674775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 218/426] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:11 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:11.676159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af16d0-8ab4-4c4e-ba8b-10ce6d13db3a None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:11.676450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af16d0-8ab4-4c4e-ba8b-10ce6d13db3a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:11.676732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af16d0-8ab4-4c4e-ba8b-10ce6d13db3a None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:11.678427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 219/427] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:11.686040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af16d0-8ab4-4c4e-ba8b-10ce6d13db3a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:11.790550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:11.795738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 209/428] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:11.798569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc5eb-6096-478d-a5b2-583a433e4b10 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:11.798933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc5eb-6096-478d-a5b2-583a433e4b10 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:11.799282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc5eb-6096-478d-a5b2-583a433e4b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:11.817518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-12ccc5eb-6096-478d-a5b2-583a433e4b10 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:11.818866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-12ccc5eb-6096-478d-a5b2-583a433e4b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:11.884437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af16d0-8ab4-4c4e-ba8b-10ce6d13db3a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:11.918637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 210/429] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:11.920377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccc5eb-6096-478d-a5b2-583a433e4b10 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:11.925629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:11.954501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:11.957964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:11.958875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:11.959356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:11.961555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:11.962096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:11.963780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:11.971493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 220/430] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:11 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=acffa214db594ed9803f8f5376b15d42 => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:11.972663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:12.211437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccc5eb-6096-478d-a5b2-583a433e4b10 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:12.234969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee95bdb-10bf-40ee-a6b7-fd351e7e9cfd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:12.285010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:12.286551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.286839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.287149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.291551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:12.313046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:12.314051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:12.315575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.316051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.316520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] PATH_INFO: `/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.319965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf43a1fb0074f3c8f0a85646b793ebc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:12.323467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:12.326793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:12.327740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 07:49:12.328580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:12.329246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 211/431] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:11 2022] GET /identity/v3/projects/6bf43a1fb0074f3c8f0a85646b793ebc => generated 298 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:12.329317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 221/432] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:12.335069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:12.337636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:12.376247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:12.473830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2ac40-e9bf-4e57-a8b5-2e26d81f9671 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:12.518320 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52817123-0df0-42ae-aeaa-9d4d35c78310 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:12.529491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a81ae7-1637-40dd-b6fb-daa0671a4db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:12.530826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a81ae7-1637-40dd-b6fb-daa0671a4db6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.531248 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a81ae7-1637-40dd-b6fb-daa0671a4db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.531617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a81ae7-1637-40dd-b6fb-daa0671a4db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.533132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a81ae7-1637-40dd-b6fb-daa0671a4db6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:12.533132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a81ae7-1637-40dd-b6fb-daa0671a4db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:12.534910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a81ae7-1637-40dd-b6fb-daa0671a4db6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:12.550126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 222/433] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:49:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:12.551555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e2179-d187-4ae3-9b26-d70dccfaf455 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.551850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e2179-d187-4ae3-9b26-d70dccfaf455 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.552133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e2179-d187-4ae3-9b26-d70dccfaf455 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.553697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 223/434] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:12.564188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd09a-a46d-4c8a-95ff-516e689d5087 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.564701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd09a-a46d-4c8a-95ff-516e689d5087 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.565257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd09a-a46d-4c8a-95ff-516e689d5087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.584893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:12.587610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.587893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.588345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.590997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:12.592462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:12.597313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-54abd09a-a46d-4c8a-95ff-516e689d5087 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:12.597580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 212/435] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:12 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:12.597671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-54abd09a-a46d-4c8a-95ff-516e689d5087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:12.597746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:12.692470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:12.746519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 224/436] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:12.747420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd09a-a46d-4c8a-95ff-516e689d5087 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:12.769872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a464911-992f-4c1d-87ad-0f171c8696e2 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:12.773316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:12.816072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:12.816766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.817105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.817389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.928240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:12.931667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd09a-a46d-4c8a-95ff-516e689d5087 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:12.932430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:12.987583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:12.988989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:12.989444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:12.990285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:12.993169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:12.993690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:12.997421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:13.005692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 225/437] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:12 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=6bf43a1fb0074f3c8f0a85646b793ebc&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:13.006744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:13.072931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:13.178860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 213/438] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:13.188119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:13.196851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:13.242814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf9285-e080-4463-8958-698623a6fa9c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:13.315863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:13.317836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.318519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.319020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.342786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:13.353398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:13.354818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:13.355958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:13.359472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 226/439] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:13.377591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:13.402536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8f44b-1e2c-4d07-afdb-304c0512d20c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:13.457160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:13.458744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.459265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.459758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.462701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:13.465605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:13.468896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 214/440] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:13 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:13.476208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:13.558251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:13.558685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544768-b69f-49b5-97ec-a9d2f85dd033 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:13.608607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:13.616415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:13.618153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.618660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.619161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.620578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:13.621165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:13.623792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:13.642093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 227/441] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:13.651790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:13.679119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831787de-1a2f-4393-a9f9-046ede7ed51d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:13.728959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963c4ab-c53a-4c08-b413-7f686ac533d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:13.729868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963c4ab-c53a-4c08-b413-7f686ac533d1 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.730158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963c4ab-c53a-4c08-b413-7f686ac533d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.730439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963c4ab-c53a-4c08-b413-7f686ac533d1 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.731656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963c4ab-c53a-4c08-b413-7f686ac533d1 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:13.733167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963c4ab-c53a-4c08-b413-7f686ac533d1 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:13.734823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 215/442] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:13 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:13.737852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f17cd5-c196-4bc0-8d1d-e759026c9157 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.738364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f17cd5-c196-4bc0-8d1d-e759026c9157 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.738865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f17cd5-c196-4bc0-8d1d-e759026c9157 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.741253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 216/443] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:13.743183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f17cd5-c196-4bc0-8d1d-e759026c9157 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:13.811104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d230bc-55b6-4ef2-8396-f83346bf60f0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:13.861341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:13.862021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.862373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.862770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.874457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:13.879647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f17cd5-c196-4bc0-8d1d-e759026c9157 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:13.934498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb198f-4178-428f-97bf-02a480c402a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:13.935499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb198f-4178-428f-97bf-02a480c402a0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.935924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb198f-4178-428f-97bf-02a480c402a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.936362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb198f-4178-428f-97bf-02a480c402a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.937793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb198f-4178-428f-97bf-02a480c402a0 None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=acffa214db594ed9803f8f5376b15d42, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:13.938239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb198f-4178-428f-97bf-02a480c402a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:13.940249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb198f-4178-428f-97bf-02a480c402a0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:13.948753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 217/444] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:13 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=acffa214db594ed9803f8f5376b15d42 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:13.950150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4680b-8612-49ba-8deb-a1a8e590be44 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:13.950437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4680b-8612-49ba-8deb-a1a8e590be44 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:13.950716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4680b-8612-49ba-8deb-a1a8e590be44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:13.958098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:13.973997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:13.977267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:13.988556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-84b4680b-8612-49ba-8deb-a1a8e590be44 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:13.990433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-84b4680b-8612-49ba-8deb-a1a8e590be44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:14.103986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 218/445] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:14.144039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4680b-8612-49ba-8deb-a1a8e590be44 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:14.162776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 228/446] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:14.168406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:14.316871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:14.330226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:14.332033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f27d5-5851-46b3-92ec-29c1e9e15daa None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:14.344062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4680b-8612-49ba-8deb-a1a8e590be44 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:14.393245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:14.393897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:14.394635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:14.394997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:14.395297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:14.395443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] PATH_INFO: `/v3/projects/c4e7f90bb9da44679fd899e1c8b90999` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:14.395816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:14.396386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:14.397502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] RBAC: Authorizing `identity:get_project(project_id=c4e7f90bb9da44679fd899e1c8b90999)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:14.401091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:14.404258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 229/447] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:14 2022] GET /identity/v3/projects/c4e7f90bb9da44679fd899e1c8b90999 => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:14.410885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:14.418126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:14.430916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:14.432627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:14.434181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:14.439235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 219/448] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:49:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:14.442679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:14.566467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42684600-678d-4787-a706-e1032d4f54ff None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:14.582793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1ac72-8c6a-4a21-88f3-6265125199bb None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:14.609082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:14.610250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:14.610756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:14.611260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:14.612704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:14.613309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:14.615939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:14.624351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:14.625239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:14.625522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:14.625798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] PATH_INFO: `/v3/groups/2d41a6dee10c49a49b9dd559b081c7c0` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:14.627032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] RBAC: Authorizing `identity:get_group(group_id=2d41a6dee10c49a49b9dd559b081c7c0)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:14.628481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:14.630357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 230/449] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:14 2022] GET /identity/v3/groups/2d41a6dee10c49a49b9dd559b081c7c0 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:14.633832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 220/450] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:14.637495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:14.640662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:14.666597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:14.675441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:14.764561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81565d6-5825-4327-b020-39a1493e67d0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:14.779762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc34-6d48-4301-abae-80da7cf74d5b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:14.820747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:14.821730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:14.821940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:14.822388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:14.822388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:14.822825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:14.823344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:14.823836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:14.825482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] RBAC: Adding query filter params (group.id=2d41a6dee10c49a49b9dd559b081c7c0, role.id=fe0d00714612495091430762d114121a, scope.project.id=c4e7f90bb9da44679fd899e1c8b90999) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:14.826045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:14.828793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:14.838207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 231/451] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:14 2022] GET /identity/v3/role_assignments?group.id=2d41a6dee10c49a49b9dd559b081c7c0&scope.project.id=c4e7f90bb9da44679fd899e1c8b90999&role.id=fe0d00714612495091430762d114121a => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:14.912913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:14.915359 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:14.939599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:15.066709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 221/452] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:49:15.076456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:15.178720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:15.187612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d755b7-afa7-4402-a4c4-e251df85618e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:15.223933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:15.224763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:15.225116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:15.225444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] PATH_INFO: `/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:15.227452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] RBAC: Authorizing `identity:get_project(project_id=114512b9b8fe40f4a5d4d34ea099a4ba)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:15.229220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:15.231046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 232/453] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:49:15 2022] GET /identity/v3/projects/114512b9b8fe40f4a5d4d34ea099a4ba => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:15.236803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:15.357366 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd6d5c-482a-4693-b51b-a08f873beac1 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:15.375842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:15.396861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2f1d8-b48b-4033-aaa9-dfb3669b2ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:15.397841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2f1d8-b48b-4033-aaa9-dfb3669b2ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:15.398166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2f1d8-b48b-4033-aaa9-dfb3669b2ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:15.398568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2f1d8-b48b-4033-aaa9-dfb3669b2ae2 None admin] PATH_INFO: `/v3/groups/6a2574d6b2ce498eac0349172d4f6d12` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:15.400602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2f1d8-b48b-4033-aaa9-dfb3669b2ae2 None admin] RBAC: Authorizing `identity:get_group(group_id=6a2574d6b2ce498eac0349172d4f6d12)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:15.402257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2f1d8-b48b-4033-aaa9-dfb3669b2ae2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:15.404048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 222/454] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jun 29 07:49:15 2022] GET /identity/v3/groups/6a2574d6b2ce498eac0349172d4f6d12 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:15.410205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:15.528239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd88a-2b65-4f51-9739-d5a302054787 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:15.564714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f2b6b-6c19-49bf-887c-525048b4ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:15.565407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f2b6b-6c19-49bf-887c-525048b4ded5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:15.565694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f2b6b-6c19-49bf-887c-525048b4ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:15.565972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f2b6b-6c19-49bf-887c-525048b4ded5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:15.566810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f2b6b-6c19-49bf-887c-525048b4ded5 None admin] RBAC: Adding query filter params (group.id=6a2574d6b2ce498eac0349172d4f6d12, role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:15.567125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f2b6b-6c19-49bf-887c-525048b4ded5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:15.568591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f2b6b-6c19-49bf-887c-525048b4ded5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:15.576284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 233/455] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jun 29 07:49:15 2022] GET /identity/v3/role_assignments?group.id=6a2574d6b2ce498eac0349172d4f6d12&scope.project.id=114512b9b8fe40f4a5d4d34ea099a4ba&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:18.806334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fe2d4-2897-4256-9bde-11c3d1ffcb35 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:18.807501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fe2d4-2897-4256-9bde-11c3d1ffcb35 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:18.808288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fe2d4-2897-4256-9bde-11c3d1ffcb35 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:18.811668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 223/456] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:18.822298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62e779-3036-4be3-8777-ef45a8ffc18d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:18.823774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62e779-3036-4be3-8777-ef45a8ffc18d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:18.824565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62e779-3036-4be3-8777-ef45a8ffc18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:18.869534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7c62e779-3036-4be3-8777-ef45a8ffc18d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:18.872049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7c62e779-3036-4be3-8777-ef45a8ffc18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:18.890180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d681-13f7-4dad-8af1-c3ad2f7b05fd None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:18.890806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d681-13f7-4dad-8af1-c3ad2f7b05fd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:18.891316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d681-13f7-4dad-8af1-c3ad2f7b05fd None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:18.893401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 224/457] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:18.899830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec4232-8a0c-4a76-aea6-fa92af87b441 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:18.900541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec4232-8a0c-4a76-aea6-fa92af87b441 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:18.901057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec4232-8a0c-4a76-aea6-fa92af87b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:18.927153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:18.946578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-fcec4232-8a0c-4a76-aea6-fa92af87b441 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:18.949166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fcec4232-8a0c-4a76-aea6-fa92af87b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:19.031120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 234/458] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:19.057488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62e779-3036-4be3-8777-ef45a8ffc18d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:19.110793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 225/459] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:19.118860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d7be0-b68e-48a8-aad8-d1eea1c872e7 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:19.119382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d7be0-b68e-48a8-aad8-d1eea1c872e7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:19.119808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d7be0-b68e-48a8-aad8-d1eea1c872e7 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:19.121721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 226/460] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:19.129373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5305b7-bcf1-4747-b7d8-329f7b156228 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:19.130183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5305b7-bcf1-4747-b7d8-329f7b156228 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:19.130376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5305b7-bcf1-4747-b7d8-329f7b156228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:19.161924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ad5305b7-bcf1-4747-b7d8-329f7b156228 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:19.163476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ad5305b7-bcf1-4747-b7d8-329f7b156228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:19.244159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62e779-3036-4be3-8777-ef45a8ffc18d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:19.274401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 227/461] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:19.275924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5305b7-bcf1-4747-b7d8-329f7b156228 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:19.305300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:19.306950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:19.307742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:19.308219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:19.327240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:19.330089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:19.353335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:19.366261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:19.368941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:19.370878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:19.370878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:19.370878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:19.370878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:19.374176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 235/462] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:19.375920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:19.403239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:19.449487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5305b7-bcf1-4747-b7d8-329f7b156228 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:19.514635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:19.515914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:19.516750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:19.517455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:19.538371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:19.541694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:19.581715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:19.583335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:19.584960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:19.584960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:19.584960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:19.584960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:19.588346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c0613-e9f7-4b1d-a479-8e7953d25f4f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:19.589277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 228/463] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:19.590957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:19.658804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:19.660810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:19.661709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:19.662403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:19.685448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:19.688908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:19.727434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:19.729340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:19.731005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:19.731005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:19.731005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:19.731005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:19.734689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 236/464] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:19.736442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:19.786528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:19.797865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50aee9c-1496-496a-9f57-8cbebc285c80 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:19.850577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:19.876202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:19.877598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:19.878222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:19.878854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:19.881032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:19.881388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:19.885638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:19.905299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad1fd2-31a2-4201-a049-6ca90bd03fb9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:19.925395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 229/465] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:19.927462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:19.946736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:19.947794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:19.948367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:19.948866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:19.949968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:19.950496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:19.952246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:19.966567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 237/466] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:19.967772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:20.088039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a559b3-df3f-46e9-8e6a-d9b52e2598e3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:20.115719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:20.130661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae50b4-5211-49ce-86e4-fb805ad82f50 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:20.135459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:20.136272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.136625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.136998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.137809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:20.139493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:20.166980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 238/467] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:49:19 2022] POST /identity/v3/users => generated 246 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:49:20.168247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:20.201460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011769e-f325-43ff-99b1-2bc249a079ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:20.203232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011769e-f325-43ff-99b1-2bc249a079ec None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.204020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011769e-f325-43ff-99b1-2bc249a079ec None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.204757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011769e-f325-43ff-99b1-2bc249a079ec None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.206786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011769e-f325-43ff-99b1-2bc249a079ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:20.207641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011769e-f325-43ff-99b1-2bc249a079ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:20.211518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011769e-f325-43ff-99b1-2bc249a079ec None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:20.236369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 230/468] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:20.239266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f40b1-6f46-4921-b2b0-374ef05467be None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.240108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f40b1-6f46-4921-b2b0-374ef05467be None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.241013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f40b1-6f46-4921-b2b0-374ef05467be None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.244345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 231/469] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:20.246196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f40b1-6f46-4921-b2b0-374ef05467be None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:20.284478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:20.300010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742ca65-e886-45ba-ad51-af026c462168 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:20.343713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b56b2-d11a-48ec-9869-099931cc38a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:20.344410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b56b2-d11a-48ec-9869-099931cc38a6 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.344700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b56b2-d11a-48ec-9869-099931cc38a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.345010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b56b2-d11a-48ec-9869-099931cc38a6 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.345756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b56b2-d11a-48ec-9869-099931cc38a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:20.347502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b56b2-d11a-48ec-9869-099931cc38a6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:20.373800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 239/470] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:49:20 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:49:20.375184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1067287-0e3a-406d-829c-78c612d53c95 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.375487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1067287-0e3a-406d-829c-78c612d53c95 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.375750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1067287-0e3a-406d-829c-78c612d53c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.396279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-a1067287-0e3a-406d-829c-78c612d53c95 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:20.397893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a1067287-0e3a-406d-829c-78c612d53c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:20.437393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:20.462626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f40b1-6f46-4921-b2b0-374ef05467be None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:20.487390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 240/471] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:20.515103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1067287-0e3a-406d-829c-78c612d53c95 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:20.543939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:20.544899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.545589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.546242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.547912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:20.551315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:20.600181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 232/472] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:49:20 2022] POST /identity/v3/users => generated 248 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:49:20.669562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1067287-0e3a-406d-829c-78c612d53c95 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:20.716716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c204b-12e6-4eba-b5f7-b18a048cd8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:20.717586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c204b-12e6-4eba-b5f7-b18a048cd8ac None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.717874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c204b-12e6-4eba-b5f7-b18a048cd8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.718176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c204b-12e6-4eba-b5f7-b18a048cd8ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.733284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c204b-12e6-4eba-b5f7-b18a048cd8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:20.734920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c204b-12e6-4eba-b5f7-b18a048cd8ac None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-8c8c204b-12e6-4eba-b5f7-b18a048cd8ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:20.757053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:20.761453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:20.764599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:20.764599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:20.764599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:20.764599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:20.764599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 241/473] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:20.767653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:20.814927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f5c0-92ee-4bc7-b2ac-0bacf6019dd1 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.815188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f5c0-92ee-4bc7-b2ac-0bacf6019dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.815512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f5c0-92ee-4bc7-b2ac-0bacf6019dd1 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.817371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 242/474] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:20.826411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a3056-9047-4f20-94df-4303b4a5e2a3 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:20.826815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a3056-9047-4f20-94df-4303b4a5e2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:20.826966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a3056-9047-4f20-94df-4303b4a5e2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:20.851805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-729a3056-9047-4f20-94df-4303b4a5e2a3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:20.853094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-729a3056-9047-4f20-94df-4303b4a5e2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:20.869011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:20.895726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:20.966132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d84d-9c67-46d7-92ce-2e136c64e2dd None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:20.985308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 243/475] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:21.013967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a3056-9047-4f20-94df-4303b4a5e2a3 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:21.024813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:21.025608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:21.025895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:21.026168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:21.026931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:21.027239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:21.028672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:21.044853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 233/476] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:21.051689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:21.141260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a3056-9047-4f20-94df-4303b4a5e2a3 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:21.183899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:21.185182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:21.188408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:21.189032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:21.189956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:21.191432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:21.213839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.sql.core [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 07:49:21.213839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 07:49:21.213839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [parameters: {'id': '5dac636f21d245dc8f8185542bbc017a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 07:49:21.213839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80107) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application conn.query(q) Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result.read() Jun 29 07:49:21.246459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return ref.to_dict() Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 07:49:21.247438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application next(self.gen) Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application yield resource Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application next(self.gen) Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise self.value Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application session.commit() Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 07:49:21.248357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.session.flush() Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application compat.raise_( Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 07:49:21.249274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 07:49:21.250168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application conn.query(q) Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result.read() Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 07:49:21.250968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [parameters: {'id': '5dac636f21d245dc8f8185542bbc017a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:21.251725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 07:49:21.252509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:21.262061 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 244/477] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jun 29 07:49:21 2022] POST /identity/v3/roles => generated 144 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:21.268283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:21.270188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e49f82-eeea-4840-9b79-c3177871584c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:21.284724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:21.329915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:21.347762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:21.348819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:21.349327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:21.349847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:21.350776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:21.353363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:21.422294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 234/478] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:49:21 2022] POST /identity/v3/users => generated 249 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:49:21.495951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b28f6-6887-4124-a81b-3f84d466c0bf None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:21.567227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0a312-8ef6-4aa5-9701-ae0be431c288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:21.568087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0a312-8ef6-4aa5-9701-ae0be431c288 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:21.568381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0a312-8ef6-4aa5-9701-ae0be431c288 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:21.568666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0a312-8ef6-4aa5-9701-ae0be431c288 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:21.582649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0a312-8ef6-4aa5-9701-ae0be431c288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-3ea0a312-8ef6-4aa5-9701-ae0be431c288 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:21.603134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:21.604155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:21.605153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 245/479] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:49:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:21.612682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:21.792456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1d34-f1ee-433a-8f78-4217c7e05528 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:21.843772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9711d61-7bb0-464d-bf67-4734719003bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:21.844951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9711d61-7bb0-464d-bf67-4734719003bd None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:21.845560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9711d61-7bb0-464d-bf67-4734719003bd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:21.846062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9711d61-7bb0-464d-bf67-4734719003bd None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:21.847486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9711d61-7bb0-464d-bf67-4734719003bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:21.848060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9711d61-7bb0-464d-bf67-4734719003bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:21.850720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9711d61-7bb0-464d-bf67-4734719003bd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:21.881709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 235/480] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:23.582534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3235f8-09cb-419f-9c26-f901c524002f None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:23.583070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3235f8-09cb-419f-9c26-f901c524002f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:23.583070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3235f8-09cb-419f-9c26-f901c524002f None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:23.584660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 246/481] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:23.591457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ccc9-c599-4333-8937-809e97f4a032 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:23.592133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ccc9-c599-4333-8937-809e97f4a032 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:23.592133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ccc9-c599-4333-8937-809e97f4a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:23.620659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f649ccc9-c599-4333-8937-809e97f4a032 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:23.621954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f649ccc9-c599-4333-8937-809e97f4a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:23.692253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:23.722781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 236/482] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:23.723283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a002b0-682e-46c3-98c3-e8a2c31b4460 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:23.723772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a002b0-682e-46c3-98c3-e8a2c31b4460 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:23.724247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a002b0-682e-46c3-98c3-e8a2c31b4460 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:23.725843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 247/483] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:49:23.742849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acf20e-bae4-4c7e-9b1a-a19668ea49b6 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:23.743181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acf20e-bae4-4c7e-9b1a-a19668ea49b6 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:23.743471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acf20e-bae4-4c7e-9b1a-a19668ea49b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:23.756693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a002b0-682e-46c3-98c3-e8a2c31b4460 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:23.770815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-09acf20e-bae4-4c7e-9b1a-a19668ea49b6 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:23.772627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-09acf20e-bae4-4c7e-9b1a-a19668ea49b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:23.905986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 237/484] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:23.919811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:23.947761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acf20e-bae4-4c7e-9b1a-a19668ea49b6 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:24.040453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a002b0-682e-46c3-98c3-e8a2c31b4460 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:24.066008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acf20e-bae4-4c7e-9b1a-a19668ea49b6 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:24.078717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:24.110279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:24.114269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:24.114825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:24.115291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:24.137770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:24.149312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:24.150562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:24.151337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:24.155384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:24.156648 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:24.157169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:24.157654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:24.161658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 238/485] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:24.172434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:24.195609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:24.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:24.227158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:24.228055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:24.233286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 248/486] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:24.250715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:24.305966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9025-04e7-4105-a3fb-c563835a4538 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:24.344766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6f0b3-0c5b-49d9-8794-833681087658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:24.345976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f0b3-0c5b-49d9-8794-833681087658 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:24.346487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f0b3-0c5b-49d9-8794-833681087658 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:24.346962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f0b3-0c5b-49d9-8794-833681087658 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:24.347926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6f0b3-0c5b-49d9-8794-833681087658 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:24.348441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6f0b3-0c5b-49d9-8794-833681087658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:24.350206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6f0b3-0c5b-49d9-8794-833681087658 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:24.361125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 239/487] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:24.362833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176272a-5486-4a6d-9143-e20946c137a0 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:24.363350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176272a-5486-4a6d-9143-e20946c137a0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:24.363828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176272a-5486-4a6d-9143-e20946c137a0 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:24.365493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 240/488] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:24.370682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176272a-5486-4a6d-9143-e20946c137a0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:24.429434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:24.470515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a53e-14ae-4dce-9c1a-c39355fecef0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:24.525158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:24.542202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441546-aa90-4a54-96b7-7a34a17654bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:24.543279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441546-aa90-4a54-96b7-7a34a17654bc None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:24.544261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441546-aa90-4a54-96b7-7a34a17654bc None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:24.544736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441546-aa90-4a54-96b7-7a34a17654bc None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:24.546102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441546-aa90-4a54-96b7-7a34a17654bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:24.547015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441546-aa90-4a54-96b7-7a34a17654bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:24.550066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441546-aa90-4a54-96b7-7a34a17654bc None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:24.568117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 249/489] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:24.571032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2566e74-42c8-4a7b-bcb7-58d8bfcda05d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:24.571512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2566e74-42c8-4a7b-bcb7-58d8bfcda05d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:24.572001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2566e74-42c8-4a7b-bcb7-58d8bfcda05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:24.591909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176272a-5486-4a6d-9143-e20946c137a0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:24.602661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f2566e74-42c8-4a7b-bcb7-58d8bfcda05d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:24.604579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f2566e74-42c8-4a7b-bcb7-58d8bfcda05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:24.656309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:24.658322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:24.659159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:24.659929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:24.681328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:24.684324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:24.718204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:24.719939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:24.721609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:24.721609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:24.721609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:24.721609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:24.731057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 241/490] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:24.733001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:24.752174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 250/491] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:24.753929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2566e74-42c8-4a7b-bcb7-58d8bfcda05d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:24.928203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:24.958492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2566e74-42c8-4a7b-bcb7-58d8bfcda05d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:24.961775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cd5a-5a04-4787-abd0-43c1bfc061ea None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:24.995227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:25.020344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.021440 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.021881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.023682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:25.024226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:25.026887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:25.052069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 251/492] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:25.052287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2fad-9026-4998-bb7a-d0b131ec6616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.052407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.052500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2fad-9026-4998-bb7a-d0b131ec6616 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.052500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2fad-9026-4998-bb7a-d0b131ec6616 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.052500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2fad-9026-4998-bb7a-d0b131ec6616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.068848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2fad-9026-4998-bb7a-d0b131ec6616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:25.071308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2fad-9026-4998-bb7a-d0b131ec6616 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-192d2fad-9026-4998-bb7a-d0b131ec6616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:25.094300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:25.095479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:25.096471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:25.096471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:25.096471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:25.096471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:25.096471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 242/493] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:25.097540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66a6d-c662-4b34-8927-1ec213b02c49 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.097540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66a6d-c662-4b34-8927-1ec213b02c49 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.097540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66a6d-c662-4b34-8927-1ec213b02c49 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.099221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 243/494] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:25.099661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66a6d-c662-4b34-8927-1ec213b02c49 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.200035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495e0fa-4fae-4dac-a2d0-401931298b71 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:25.241428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.242789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.243218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.243696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.251664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66a6d-c662-4b34-8927-1ec213b02c49 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:25.256997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:25.267022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:25.268081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:25.268878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:25.273170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 252/495] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:25.274568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.307414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662012d-9eaa-4fcd-a4d9-490820fc5f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.308809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662012d-9eaa-4fcd-a4d9-490820fc5f3f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.309473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662012d-9eaa-4fcd-a4d9-490820fc5f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.309926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662012d-9eaa-4fcd-a4d9-490820fc5f3f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.314209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:25.322461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662012d-9eaa-4fcd-a4d9-490820fc5f3f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:25.324100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662012d-9eaa-4fcd-a4d9-490820fc5f3f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:25.331669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-f662012d-9eaa-4fcd-a4d9-490820fc5f3f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:25.348311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:25.350132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 244/496] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a2792-b9d9-4d9e-bc00-a06523788383 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.359484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a2792-b9d9-4d9e-bc00-a06523788383 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.360480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a2792-b9d9-4d9e-bc00-a06523788383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.383668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-861a2792-b9d9-4d9e-bc00-a06523788383 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:25.385586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-861a2792-b9d9-4d9e-bc00-a06523788383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:25.427082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6323062-44d8-4ffe-890b-06d970e9aadb None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:25.464962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.465569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.465857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.466138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.466877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:25.467187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:25.468597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:25.474141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 245/497] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:25.475540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a2792-b9d9-4d9e-bc00-a06523788383 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.488881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 253/498] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:25.494060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.610732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a2792-b9d9-4d9e-bc00-a06523788383 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:25.616010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:25.620047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f40a3-c6d0-4e90-8b07-684badf830a5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:25.642312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:25.652570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043cd5-2e67-438f-887e-48d279729ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.653314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043cd5-2e67-438f-887e-48d279729ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.653602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043cd5-2e67-438f-887e-48d279729ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.653883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043cd5-2e67-438f-887e-48d279729ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.654685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043cd5-2e67-438f-887e-48d279729ee4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:25.654999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043cd5-2e67-438f-887e-48d279729ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:25.656463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043cd5-2e67-438f-887e-48d279729ee4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:25.663995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.664692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.665015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.665283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.666154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:25.666463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:25.666646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 246/499] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:25.667991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:25.668193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b31be-1f7a-48ab-9792-8115601f558c None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.668331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b31be-1f7a-48ab-9792-8115601f558c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.668606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b31be-1f7a-48ab-9792-8115601f558c None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.676285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 247/500] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:25.676285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b31be-1f7a-48ab-9792-8115601f558c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.714705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 254/501] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:25.715733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.798565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b31be-1f7a-48ab-9792-8115601f558c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:25.834976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5ed45-1cd0-4442-a5f3-4ded2b96723d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:25.839817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.840630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.840967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.841275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.849590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:25.851620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:25.867754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:25.869324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:25.870177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 248/502] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:25.870832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:25.883685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d057146-c0cf-4924-aee0-786c8405bdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:25.884760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d057146-c0cf-4924-aee0-786c8405bdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.885218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d057146-c0cf-4924-aee0-786c8405bdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.885617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d057146-c0cf-4924-aee0-786c8405bdd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.901755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d057146-c0cf-4924-aee0-786c8405bdd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-9d057146-c0cf-4924-aee0-786c8405bdd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:25.915930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:25.917272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:25.918713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 255/503] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:25.919981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed8ab9-97d5-48a4-9f4e-fb1ba3414924 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:25.920110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed8ab9-97d5-48a4-9f4e-fb1ba3414924 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:25.920284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed8ab9-97d5-48a4-9f4e-fb1ba3414924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:25.943432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-a5ed8ab9-97d5-48a4-9f4e-fb1ba3414924 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:25.944656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a5ed8ab9-97d5-48a4-9f4e-fb1ba3414924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:25.951518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:25.963358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:26.025521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 256/504] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:26.026286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed8ab9-97d5-48a4-9f4e-fb1ba3414924 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:26.035894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1100-0b34-4f87-81f3-141afc80aa7c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:26.084834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:26.086491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:26.087346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:26.087714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:26.100625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:26.102690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:26.125429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:26.127196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:26.128830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:26.128830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:26.128830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:26.128830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:26.128830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 249/505] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:26.129411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:26.166445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed8ab9-97d5-48a4-9f4e-fb1ba3414924 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:26.219735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:26.220523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:26.220871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:26.221233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:26.246259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:26.279257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:26.282704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab96a0-9c62-44b3-a716-54f84756eab6 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:26.324564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:26.325337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:26.326086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:26.326239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:26.326492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:26.326858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:26.327778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:26.328164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:26.329737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:26.370928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 250/506] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jun 29 07:49:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:26.372059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:26.510705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 257/507] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:26.512291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:26.525282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b0346-a0ad-4958-bd0d-c6b9e3a9f5b5 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:26.575803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:26.577018 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:26.577500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:26.578010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:26.579457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:26.580010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:26.582684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:26.584882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:26.603856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 251/508] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:26.603856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:26.640871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:26.645063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d498a-7533-40c0-8679-6ba20d9f0a19 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:26.691505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:26.692865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:26.693401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:26.693892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:26.713358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:26.716095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:26.737479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be98f-579c-4564-a87e-0cfc32e421f6 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:26.750885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:26.752672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:26.754585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:26.754585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:26.754585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:26.754585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:26.757180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 258/509] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:26.759413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:26.782223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:26.783304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:26.783738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:26.784184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:26.785477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:26.785992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:26.788362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:26.811888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 252/510] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:26.813142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:26.908838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:26.911980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376449c2-6fa2-43f8-aea2-79ca28aa7e8f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:26.946440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:26.955899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:26.956585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:26.956868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:26.957185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:26.978512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01cb2d-0bd0-4a4d-a29d-cc34826a0227 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.029664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.030461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.030750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.031020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.042992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.044621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.044915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.046415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:27.064396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:27.066098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:27.068078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:27.068078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:27.068078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:27.068078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:27.068078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 253/511] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:27.068078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.167511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:27.177065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 259/512] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:27.178041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.216256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c5161-3291-48b2-94b2-0c91587bc03f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.252530 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:27.260262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.261597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.262053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.262390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.280764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.283382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:27.307908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:27.309758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:27.311009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:27.311009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:27.311009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:27.311009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:27.311009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 254/513] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:27.311009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.326068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d014e-aba5-4678-b03b-03762800f1ed None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.366975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.367637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.367976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.368185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.368926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:27.369270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.370688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.384418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 260/514] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:27.385335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.458462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60883f41-f79c-45b4-83eb-17a175fcc819 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.462704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:27.505051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.505863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.506148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.506428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.516045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.517558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.518750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ea6b1-4583-4832-8a26-27c8ae6805fb None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:27.533119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:27.535098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 255/515] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:27.536866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.565165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.565835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.566120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.566399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.567144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:27.567446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.568863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.583177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 261/516] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:27.584036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.622654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:27.695651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1de291-022b-497e-bfcf-610c4838fa74 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.712588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae26ddc-54c7-4bff-b870-141ced6bcfb7 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.731682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.732499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.732785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.733088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.746096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.747593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.748217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:27.766649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:27.767890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:27.770289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:27.770289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:27.770289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:27.770289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:27.770289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 256/517] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:27.770289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.773452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.774604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.775108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.775603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.776365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:27.776672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.778102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.797200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 262/518] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:27.799347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:27.881861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:27.889276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef078c6e-c962-4c84-aa65-3c72faf63784 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.930436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:27.931265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:27.931628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:27.932078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:27.933090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:27.934846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:27.952871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ee30f-effd-4c3d-8e66-96643b3f963f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:27.958033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 257/519] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:49:27 2022] POST /identity/v3/users => generated 247 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:49:27.958033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.005569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.006526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.006804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.007113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.008779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:28.009268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.010823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.033046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 263/520] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:28.033946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.090027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:28.122912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26785-d459-4e7e-845a-465063a2566c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:28.181149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.182196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.182609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.183081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.187386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629ee1-4b76-4c63-9db2-c09ffedc735b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:28.192091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.193623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:28.215433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:28.217105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 258/521] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:28.218954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.238862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:28.252201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.253214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.253652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.254105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.255158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:28.256869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.260799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.286493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 264/522] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:28.288188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.409812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c744b7-1320-4270-a84f-bee85962e8da None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:28.470942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:28.477740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974db5d-b860-4438-b5f7-5fa5ece3c80f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:28.482675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.484056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.484571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.485108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.497904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.499468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:28.517084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:28.519031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 07:49:28.522171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:28.524250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 259/523] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:28.524614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.543383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.544184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.544543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.544892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.625182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:28.633916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.635424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.665728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89877f-d57b-4c3c-a90f-96caa4a2ec57 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:28.707407 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.708438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.708595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.708967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.709904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:28.710309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.712455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.741110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 260/524] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:28.741110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.768160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:28.783320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 265/525] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:28.783320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.879257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c9c80-4e39-4482-a9db-7e74d27ff223 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:28.906093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bae84-2136-46bf-a83d-1202f38b3f69 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:28.918127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:28.925616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.926434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.926730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.926985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.936007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.937562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:28.952770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:28.953836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:28.954766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:28.955175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:28.955266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 261/526] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:28.955266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:28.961879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:28.962269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:28.962445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:28.963350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:28.963668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:28.965238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:28.986594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 266/527] 10.222.0.43 () {56 vars in 1139 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:28.987461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:29.088809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:29.134121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62090f23-53e4-4acb-84eb-c8575c037e26 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:29.150796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896c7d0-3d1a-441d-86fe-4f94f28c451d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:29.194684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:29.195049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:29.195567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:29.195848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:29.196009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:29.196281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:29.196442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:29.196868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:29.198307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=24630ccd20794f19a138a19187348920) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:29.198801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:29.201366 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:29.217014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 262/528] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/role_assignments?user.id=24630ccd20794f19a138a19187348920&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:29.217238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:29.217351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:29.217503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:29.252712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:29.254402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:29.256013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:29.256013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:29.256013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:29.256013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:29.261546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 267/529] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:29.262655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:29.308947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:29.373799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9b13a-ef03-4cf7-9cff-7356a8ec6000 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:29.427047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:29.428182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:29.428573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:29.429426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:29.431668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:29.432270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:29.434047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:29.461242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 263/530] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:29.463152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:29.484767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbbfd0-4581-4465-b3e2-97aed4e52611 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:29.497282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:29.533390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:29.534805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:29.535091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:29.535366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:29.665527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:29.668273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:29.685462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d773967-9cb9-4033-a096-85ea2d4a5d74 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:29.719071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:29.754591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:29.756233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:29.757033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:29.758306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:29.760031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:29.760854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:29.763729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:29.787182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 264/531] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:29.789738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:29.869256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 268/532] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:29.870826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:29.889553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:29.975822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43caab0-5eef-49eb-8dd7-e252f13f530e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:30.046401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347273ab-32af-4473-b77d-8429b776060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:30.048211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347273ab-32af-4473-b77d-8429b776060e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.048948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347273ab-32af-4473-b77d-8429b776060e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.049743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347273ab-32af-4473-b77d-8429b776060e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.051542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347273ab-32af-4473-b77d-8429b776060e None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=30d3144975804e369a023316655be2d0) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:30.052344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347273ab-32af-4473-b77d-8429b776060e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:30.052844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f7c1e-9647-4654-a5eb-8c1b2560f721 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:30.056060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347273ab-32af-4473-b77d-8429b776060e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:30.066672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 265/533] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:29 2022] GET /identity/v3/role_assignments?user.id=30d3144975804e369a023316655be2d0&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:30.068808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347273ab-32af-4473-b77d-8429b776060e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:30.128383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:30.129804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.133015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.133015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.147469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:30.151712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:30.185495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:30.187179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:30.189960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 269/534] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:30.198199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:30.200002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:30.325475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347273ab-32af-4473-b77d-8429b776060e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:30.339234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:30.412701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0eec3-3acf-4d81-994b-3cafc66d7942 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:30.418764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6294421-cf18-4c8b-9c05-94a2e60d072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:30.419641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6294421-cf18-4c8b-9c05-94a2e60d072d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.419935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6294421-cf18-4c8b-9c05-94a2e60d072d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.420210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6294421-cf18-4c8b-9c05-94a2e60d072d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.449564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6294421-cf18-4c8b-9c05-94a2e60d072d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:30.451401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6294421-cf18-4c8b-9c05-94a2e60d072d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-b6294421-cf18-4c8b-9c05-94a2e60d072d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:30.485040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:30.487131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 266/535] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fad49-4bad-44ef-b549-2cb9941e40b9 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fad49-4bad-44ef-b549-2cb9941e40b9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.488771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fad49-4bad-44ef-b549-2cb9941e40b9 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.500633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 267/536] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:30.500633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fad49-4bad-44ef-b549-2cb9941e40b9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:30.517931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af4963-345a-4968-aaa6-4f61546a6cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:30.519662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af4963-345a-4968-aaa6-4f61546a6cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.520431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af4963-345a-4968-aaa6-4f61546a6cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.521250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af4963-345a-4968-aaa6-4f61546a6cc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.522541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af4963-345a-4968-aaa6-4f61546a6cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:30.523267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af4963-345a-4968-aaa6-4f61546a6cc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:30.526478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af4963-345a-4968-aaa6-4f61546a6cc0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:30.554702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 270/537] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:30.556735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbba16a-f8a0-4620-b872-61ae455a6fe1 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.557285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbba16a-f8a0-4620-b872-61ae455a6fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.557764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbba16a-f8a0-4620-b872-61ae455a6fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.578320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5cbba16a-f8a0-4620-b872-61ae455a6fe1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:30.579897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5cbba16a-f8a0-4620-b872-61ae455a6fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:30.634881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:30.651304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fad49-4bad-44ef-b549-2cb9941e40b9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:30.670921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 271/538] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:30.670921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbba16a-f8a0-4620-b872-61ae455a6fe1 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:30.716441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:30.717183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.717469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.717746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.718490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:30.718797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:30.720423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:30.736195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 268/539] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:30.737286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:30.772111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:30.853866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbba16a-f8a0-4620-b872-61ae455a6fe1 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:30.891070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:30.891742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.892069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.892342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.893263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a267b6f64f5f49b8b3f5837e701bc216) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:30.893574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:30.895032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:30.902489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 272/540] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:30 2022] GET /identity/v3/role_assignments?user.id=a267b6f64f5f49b8b3f5837e701bc216&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:30.903094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:30.919041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5bb2-f4bc-40f3-aa08-b4186ae28b9e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:30.961774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:30.963111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:30.963866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:30.964370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:30.977806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:30.986386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:30.987352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:30.988145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:30.991324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 269/541] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:30.999824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:31.019635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:31.111771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a1e72-6794-4982-8c30-5aebe37a4cdc None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:31.160854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:31.179183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6519b-a079-47e4-972e-0a2371edaa82 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:31.181138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:31.182238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:31.182741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:31.184221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:31.194071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:31.197233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:31.221171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:31.222070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:31.222920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:31.226937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 273/542] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:31.232951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:31.264697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:31.265506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:31.265779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:31.265997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:31.266805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:31.267039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:31.268818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:31.282015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 270/543] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:31.289965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:31.431927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaee22e-a63d-48ae-8893-de8433f885ba None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:31.483184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:31.502752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:31.503762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:31.504247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:31.504743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:31.505919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:31.506453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:31.508477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:31.523695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 274/544] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:31.530481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:31.575721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576b0b-2e77-4039-8ae0-3c6c868e925f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:31.599041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:31.665816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:31.667023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:31.668674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:31.669280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:31.696398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:31.697835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:31.718033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a62be-521d-4b8e-837e-50439e16cbe0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:31.732571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:31.733830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:31.734668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:31.734668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:31.734668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:31.734668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:31.734891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 271/545] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:31.739297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:31.789983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12759fc-d839-4a45-8734-0c77ce7ed140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:31.792372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12759fc-d839-4a45-8734-0c77ce7ed140 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:31.793816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12759fc-d839-4a45-8734-0c77ce7ed140 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:31.794564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12759fc-d839-4a45-8734-0c77ce7ed140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:31.797009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12759fc-d839-4a45-8734-0c77ce7ed140 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=45a558f9c7c84a05a90595fda7fccd40) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:31.798900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12759fc-d839-4a45-8734-0c77ce7ed140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:31.800601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12759fc-d839-4a45-8734-0c77ce7ed140 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:31.816756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 275/546] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:31 2022] GET /identity/v3/role_assignments?user.id=45a558f9c7c84a05a90595fda7fccd40&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:31.868659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722525da-a164-499c-9dfb-304c153d996c None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:31.869733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722525da-a164-499c-9dfb-304c153d996c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:31.869733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722525da-a164-499c-9dfb-304c153d996c None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:31.870972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 276/547] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:49:31.878056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1a976-128e-44bd-b241-39222ac44e5d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:31.878432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1a976-128e-44bd-b241-39222ac44e5d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:31.878669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1a976-128e-44bd-b241-39222ac44e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:31.914028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-aec1a976-128e-44bd-b241-39222ac44e5d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:31.919156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-aec1a976-128e-44bd-b241-39222ac44e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:31.967756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b2de2-d8c1-4b22-9619-decbfc3f9649 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:32.036303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:32.051071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:32.051762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.052052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.052333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.056362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:32.056764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:32.058184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:32.083575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 272/548] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:32.085469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 277/549] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:32.090592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:32.118000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1a976-128e-44bd-b241-39222ac44e5d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:32.157576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:32.249815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1a976-128e-44bd-b241-39222ac44e5d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:32.290731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:32.291639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.291934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.292211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.305197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:32.313568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:32.314698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:32.315921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 278/550] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:32.321712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:32.323459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950bf53-5fb1-4d61-9b88-5d7aca3aea31 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:32.435152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:32.442774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada59c27-e4c7-4d75-9e23-d176a699b7d2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:32.447506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c240d-ed34-425c-a26b-7d66b005d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:32.448402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c240d-ed34-425c-a26b-7d66b005d991 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.448867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c240d-ed34-425c-a26b-7d66b005d991 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.449371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c240d-ed34-425c-a26b-7d66b005d991 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.473724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c240d-ed34-425c-a26b-7d66b005d991 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:32.476894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c240d-ed34-425c-a26b-7d66b005d991 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:32.500258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3deb7-c4a3-41e4-a286-cea558f4ca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:32.502078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3deb7-c4a3-41e4-a286-cea558f4ca64 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.502808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3deb7-c4a3-41e4-a286-cea558f4ca64 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.503539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3deb7-c4a3-41e4-a286-cea558f4ca64 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-b33c240d-ed34-425c-a26b-7d66b005d991 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:32.503686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:32.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 273/551] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0dc22-3e5a-4bb4-93a6-d265fe26e4c3 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.508059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0dc22-3e5a-4bb4-93a6-d265fe26e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.510153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0dc22-3e5a-4bb4-93a6-d265fe26e4c3 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.513474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3deb7-c4a3-41e4-a286-cea558f4ca64 None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:32.519847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 274/552] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:32.519847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0dc22-3e5a-4bb4-93a6-d265fe26e4c3 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:32.520878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3deb7-c4a3-41e4-a286-cea558f4ca64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:32.523802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3deb7-c4a3-41e4-a286-cea558f4ca64 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:32.540183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 279/553] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:32.543000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7847ac6-e143-41a7-8034-19f353444186 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.543804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7847ac6-e143-41a7-8034-19f353444186 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.544623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7847ac6-e143-41a7-8034-19f353444186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.572456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b7847ac6-e143-41a7-8034-19f353444186 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:32.574895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b7847ac6-e143-41a7-8034-19f353444186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:32.585436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:32.644032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0dc22-3e5a-4bb4-93a6-d265fe26e4c3 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:32.689001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:32.689001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.689001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.689001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.689554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:32.689554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:32.689554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:32.689951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 280/554] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:32.691100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7847ac6-e143-41a7-8034-19f353444186 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:32.711095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 275/555] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jun 29 07:49:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:32.719866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:32.848717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:32.883860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181fbe1-1444-454e-943a-c5bf8df4d636 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:32.884382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7847ac6-e143-41a7-8034-19f353444186 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:32.889708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:32.931336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:32.933633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.934392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.935098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.954046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:32.957224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:32.965961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:32.967002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:32.967518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:32.967993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:32.991074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:32.993495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:32.995219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:32.995219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:32.995219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:32.995219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:32.998572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 281/556] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:33.000494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:33.113106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:33.116179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:33.188219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34146c9c-bc45-449b-a6db-5b765455ac0b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:33.238090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 276/557] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:33.238983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:33.242670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:33.261948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:33.263268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:33.263749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:33.264368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:33.285166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:33.298475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:33.318777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:33.321818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 282/558] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:33.325490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:33.364057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b77dc-b16b-48c5-9f75-a2fa6a91b7b7 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:33.403153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:33.403829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:33.404110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:33.404386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:33.405139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:33.405460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:33.406942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:33.424740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 277/559] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:33.424740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:33.503768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3448-b1c4-4946-964e-f3d799b84811 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:33.536813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:33.545126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa71f5-13aa-4509-97ea-1fa6e607d5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:33.546487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa71f5-13aa-4509-97ea-1fa6e607d5be None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:33.546779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa71f5-13aa-4509-97ea-1fa6e607d5be None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:33.547060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa71f5-13aa-4509-97ea-1fa6e607d5be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:33.558846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa71f5-13aa-4509-97ea-1fa6e607d5be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:33.560316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa71f5-13aa-4509-97ea-1fa6e607d5be None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-6aaa71f5-13aa-4509-97ea-1fa6e607d5be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:33.579563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:33.582003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 283/560] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669ee35-6119-413f-9e92-45d6cab5ffcd None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669ee35-6119-413f-9e92-45d6cab5ffcd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:33.583881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669ee35-6119-413f-9e92-45d6cab5ffcd None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:33.584767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 284/561] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:33.585573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669ee35-6119-413f-9e92-45d6cab5ffcd None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:33.603685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9b1e9-2291-4825-975c-dc8948aad2f4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:33.650985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da5880-9c41-4183-a93f-6172b5f7e4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:33.650985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5880-9c41-4183-a93f-6172b5f7e4fd None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:33.650985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5880-9c41-4183-a93f-6172b5f7e4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:33.650985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5880-9c41-4183-a93f-6172b5f7e4fd None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:33.651665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5880-9c41-4183-a93f-6172b5f7e4fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:33.651924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5880-9c41-4183-a93f-6172b5f7e4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:33.654669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5880-9c41-4183-a93f-6172b5f7e4fd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:33.673224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 278/562] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:33.673224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fefb0a-1b41-4301-ae62-dcceb60a7d97 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:33.673224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fefb0a-1b41-4301-ae62-dcceb60a7d97 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:33.673224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fefb0a-1b41-4301-ae62-dcceb60a7d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:33.677078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:33.701663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-22fefb0a-1b41-4301-ae62-dcceb60a7d97 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:33.703160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-22fefb0a-1b41-4301-ae62-dcceb60a7d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:33.757095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669ee35-6119-413f-9e92-45d6cab5ffcd None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:33.821072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 279/563] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:33.821281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:33.821488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fefb0a-1b41-4301-ae62-dcceb60a7d97 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:33.821566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:33.821566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:33.821566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:33.831951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:33.834513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:33.863928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:33.864960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:33.865922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 285/564] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:33.867585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:33.917625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:33.980963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fefb0a-1b41-4301-ae62-dcceb60a7d97 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.012951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419d3ab-8a50-49e2-bf88-9c232f6b8f04 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.018576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.019264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.019550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.019823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.020560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:34.020873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.022363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.030264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:34.035402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 280/565] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:34.036583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.068818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.070688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.071188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.071671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.093836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.096526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:34.130415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:34.131503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:34.137941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:34.137941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:34.137941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:34.137941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:34.137941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 286/566] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:34.137941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.165947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da4743-3924-40ff-8fc1-18dd174854f0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.194235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:34.215196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.215997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.216295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.216560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.228511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:34.236302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:34.237271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:34.238094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 281/567] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:34.238799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.275243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd08ca7-7ed8-4916-92f5-fcf908437fe0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.327433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.328566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.329098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.329586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.331276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:34.331738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.334474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.372059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 287/568] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:34.372059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.407603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e6ff-248c-40aa-8c58-804ee6e26ac1 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.410977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:34.462287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.463009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.463297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.463574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.471952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.473486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:34.493125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:34.502498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 282/569] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:34.504575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.509046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371937-80d3-4ce6-9ac1-2089767c4650 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.541356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:34.549616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.550588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.551048 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.551500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.552589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:34.553116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.555942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.574809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 288/570] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:34.575813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.635025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568b07f-2d03-44c2-b9a0-6a1df33d8c4b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.689842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446859a-b385-431d-b495-fe4131e9acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.690910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446859a-b385-431d-b495-fe4131e9acb4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.691381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446859a-b385-431d-b495-fe4131e9acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.691839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446859a-b385-431d-b495-fe4131e9acb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.692710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:34.692879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446859a-b385-431d-b495-fe4131e9acb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:34.693228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446859a-b385-431d-b495-fe4131e9acb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.695938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446859a-b385-431d-b495-fe4131e9acb4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.709019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 283/571] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:34.710963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a454ca-74ad-4e90-b7f1-61ca43f444d9 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.711253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a454ca-74ad-4e90-b7f1-61ca43f444d9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.711535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a454ca-74ad-4e90-b7f1-61ca43f444d9 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.712923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 284/572] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:34.714085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a454ca-74ad-4e90-b7f1-61ca43f444d9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.727212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943601b-109a-4d2d-b028-6fb6a422d414 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.772940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.775207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.775659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.776128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.794200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.796604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:34.822273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:34.824088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:34.833550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:34.833550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:34.833550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:34.833550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:34.833550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 289/573] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:34.833550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.853901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:34.860869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a454ca-74ad-4e90-b7f1-61ca43f444d9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:34.897199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:34.897919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:34.898201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:34.898480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:34.899276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:34.899588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:34.901055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:34.914479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 285/574] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:34.915345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:34.974197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f0920-a415-4c32-86a3-0443737b3c1e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.014396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72694e89-433f-436a-a06a-3ba71553bda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.015579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694e89-433f-436a-a06a-3ba71553bda9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.015954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694e89-433f-436a-a06a-3ba71553bda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.016331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694e89-433f-436a-a06a-3ba71553bda9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.018895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:35.027138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72694e89-433f-436a-a06a-3ba71553bda9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.029332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72694e89-433f-436a-a06a-3ba71553bda9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:35.050956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc574b-9a40-455c-9d3f-6c88f488f90a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-72694e89-433f-436a-a06a-3ba71553bda9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:35.051739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:35.052757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 290/575] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec006e-6bc9-4da0-a989-381fea5cb94a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.056726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec006e-6bc9-4da0-a989-381fea5cb94a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.058146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec006e-6bc9-4da0-a989-381fea5cb94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.080356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2bec006e-6bc9-4da0-a989-381fea5cb94a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:35.082298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2bec006e-6bc9-4da0-a989-381fea5cb94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:35.104170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.105235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.105601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.106064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.118661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.120142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:35.138332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:35.139860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:35.140858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:35.140858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:35.140858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:35.140858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:35.140858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 286/576] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:35.140858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.158449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:35.193008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 291/577] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:35.193008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec006e-6bc9-4da0-a989-381fea5cb94a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.260036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef443f-ab4d-4a4c-946c-483e7aa0db7d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.301946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.302812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.303167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.303512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.305442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:35.305829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.307722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:35.321164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 287/578] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:35.322037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.336110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec006e-6bc9-4da0-a989-381fea5cb94a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.362068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:35.375646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.376316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.376596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.376870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.377754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a411a835bce0475fae1db6e099dc2605) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:35.378057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.379523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:35.386407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 292/579] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/role_assignments?user.id=a411a835bce0475fae1db6e099dc2605&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:35.387256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.425568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:35.466426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dddaaa-21ca-414d-ae00-a77cbeb2f64f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.528571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.529209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.529494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.529766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.530639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:35.530948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.532448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:35.561020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 288/580] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:35.561020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.561322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54da0f-dd49-418c-9580-2bf92b88926d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.611114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.611815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.612097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.612383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.613226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:35.613541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.614959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:35.633856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 293/581] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:35.634759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.709414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:35.710166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7a7b7-e3fc-4859-9c84-8db15dbc8896 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.752775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:35.765263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.766087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.766373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.766653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.779627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:35.781158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ce2ce-eb98-45d8-aa20-66ec5d9e9cb3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:35.787536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:35.789186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:35.790075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:35.793378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 289/582] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:35.795353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.836646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.837618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.838104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.838576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.849079 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.850968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:35.866137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:35.867993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 294/583] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:35.870072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:35.934096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09995c7b-3eb7-4ca7-a5ba-86c9efd56b82 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:35.981387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:35.982198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:35.982483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:35.982764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:35.998416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:35.999282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.009016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70408d94-46d5-4b14-b0d0-4b57c3bca2af None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:36.020014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:36.027212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:36.028139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:36.029074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:36.029074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:36.029074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:36.029074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:36.029074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 290/584] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:36.029377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:36.058555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:36.060252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:36.061015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:36.061779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:36.076186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:36.079166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.099009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:36.111207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:36.113188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:36.115180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:36.117851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 295/585] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:49:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:36.119638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:36.180839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325621e1-afe4-4f7e-bcb4-9d3a8e30e040 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:36.224594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:36.226713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:36.227340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:36.228083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:36.229587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:36.230156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:36.232753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.247817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 291/586] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:36.248771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:36.260686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf25ed0-86c2-4abf-9b05-044832586b1b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:36.313631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:36.314266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:36.314645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:36.314921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:36.316758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:36.317099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:36.318555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.333583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 296/587] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:36.334584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:36.370627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:36.410314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed5e59-0d0f-4664-8659-30bc1cb899b8 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:36.413812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:36.465275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:36.466440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:36.466752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:36.467207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:36.468595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:36.469182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:36.471488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.496514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 292/588] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:36.497756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:36.505893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5119087-1663-469d-91ed-2ed7fe48d9db None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:36.569030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:36.570164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:36.570746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:36.571218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:36.572812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:36.573358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:36.575787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.613461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 297/589] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:49:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:36.614683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:36.668708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916cedd-6c37-4516-a695-d9e47df2d650 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:36.733122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:36.734511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:36.735043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:36.735543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:36.756759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:36.760845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.769510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:36.794593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:36.795672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:36.796698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:36.796698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:36.796698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:36.796698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:36.797253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 293/590] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:36.798891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:36.810406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d768-6507-43e9-ba27-8eb032278fb4 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:36.829829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:36.880719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:36.883080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:36.883788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:36.884286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/24630ccd20794f19a138a19187348920/roles/f36cc14a79404ebbaa216562f0f34fa1` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:36.911562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=24630ccd20794f19a138a19187348920, role_id=f36cc14a79404ebbaa216562f0f34fa1)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:36.914539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:36.945010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 298/591] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:49:36 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/24630ccd20794f19a138a19187348920/roles/f36cc14a79404ebbaa216562f0f34fa1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:36.945010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:37.030151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbf9f6-2553-4075-b65a-089067655ce7 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:37.101674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:37.102642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.103196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.104372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:37.107539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062fc30-60d1-468f-a33c-99e91d16bdd2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:37.121392 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:37.125155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:37.151751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:37.152772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:37.153733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:37.157093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 294/592] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:37.158401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:37.169927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:37.171700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.172199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.172899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:37.177816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:37.195286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:37.198581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:37.223446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:37.224555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:37.226266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:37.226266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:37.226266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:37.226266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:37.229235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 299/593] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:37.230805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:37.305984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:37.358710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dba595-7034-45b1-a3d8-9d4af6c89803 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:37.426772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:37.428318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.429212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.429944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:37.431381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:37.432201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:37.434977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:37.463856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 295/594] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:37.468525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:37.475319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f169a0-2937-491d-8f88-24bc0af5724b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:37.567506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:37.569164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.570606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.571346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:37.573120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:37.573999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:37.577949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:37.602912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 300/595] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:37.604444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:37.700599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:37.716097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc9e02-4eeb-48ee-beb1-b0c76d0975d5 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:37.771725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:37.772401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.773054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.773054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:37.773790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:37.774103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:37.775673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:37.777641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:37.806345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf08ce3-3274-4b87-9051-b0a86f84c54b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:37.809473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 296/596] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:37.810777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:37.856741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74bd2b-32f9-429b-b453-c6f62e72479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:37.857981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74bd2b-32f9-429b-b453-c6f62e72479f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.858932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74bd2b-32f9-429b-b453-c6f62e72479f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.858932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74bd2b-32f9-429b-b453-c6f62e72479f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:37.869968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74bd2b-32f9-429b-b453-c6f62e72479f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:37.871759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74bd2b-32f9-429b-b453-c6f62e72479f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-6e74bd2b-32f9-429b-b453-c6f62e72479f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:37.891074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:37.892956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:37.894848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 301/597] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:37.895981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edfb23-e08b-48de-a79c-5676fb710bcf None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.896113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edfb23-e08b-48de-a79c-5676fb710bcf None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.896870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edfb23-e08b-48de-a79c-5676fb710bcf None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:37.904662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 302/598] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:37.906042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edfb23-e08b-48de-a79c-5676fb710bcf None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:37.953986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71297a60-71c3-41f9-bd7f-d3257a62dc65 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:37.997131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:37.998061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:37.998347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:37.998626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/30d3144975804e369a023316655be2d0/roles/f36cc14a79404ebbaa216562f0f34fa1` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.018462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=30d3144975804e369a023316655be2d0, role_id=f36cc14a79404ebbaa216562f0f34fa1)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:38.020314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:38.039268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edfb23-e08b-48de-a79c-5676fb710bcf None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:38.040143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 297/599] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:49:37 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/30d3144975804e369a023316655be2d0/roles/f36cc14a79404ebbaa216562f0f34fa1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:38.047189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:38.053264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:38.094332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e5bf5-78cb-4338-83ef-207f765043e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:38.095128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5bf5-78cb-4338-83ef-207f765043e9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.095409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5bf5-78cb-4338-83ef-207f765043e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.095691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5bf5-78cb-4338-83ef-207f765043e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.103719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e5bf5-78cb-4338-83ef-207f765043e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:38.105481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e5bf5-78cb-4338-83ef-207f765043e9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a36e5bf5-78cb-4338-83ef-207f765043e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:38.121896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:38.124109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 303/600] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed33bc-0061-4d68-a726-328b97aab45b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.125905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed33bc-0061-4d68-a726-328b97aab45b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.128428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed33bc-0061-4d68-a726-328b97aab45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.145458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-83ed33bc-0061-4d68-a726-328b97aab45b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:38.146396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-83ed33bc-0061-4d68-a726-328b97aab45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:38.151745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:38.229068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 304/601] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:38.229886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed33bc-0061-4d68-a726-328b97aab45b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:38.274291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb014-40f9-48b3-9f0b-39d77a1f464a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:38.344455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:38.345543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.346470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.346956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.348463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:38.349150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:38.352690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:38.356122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:38.360700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed33bc-0061-4d68-a726-328b97aab45b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:38.380115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 298/602] 10.222.0.43 () {56 vars in 1139 bytes} [Wed Jun 29 07:49:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:38.381533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:38.425864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:38.426819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.427224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.427512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.428963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:38.429352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:38.431159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:38.447455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 305/603] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:38.448340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:38.574196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c2b0b-43c8-443a-a3a0-5c1d5b2ecbc0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:38.574196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:38.599927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6911-4f9a-43b2-bbbb-af57f113b25a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:38.610344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:38.611302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.611640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.612031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.623830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:38.633152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:38.637538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 299/604] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:38.642428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:38.651568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:38.651568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.651568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.651568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.652444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:38.652677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:38.672913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:38.676394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:38.689985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:38.689985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:38.689985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:38.689985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:38.689985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 306/605] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:38.689985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:38.690831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:38.759695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411aa6d7-cc37-4367-be24-82185fe20a51 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:38.793534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90378b51-0bb9-4eb7-b060-74c389e96c97 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:38.800321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:38.801313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.801606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.801886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/a267b6f64f5f49b8b3f5837e701bc216/roles/f36cc14a79404ebbaa216562f0f34fa1` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.819193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=a267b6f64f5f49b8b3f5837e701bc216, role_id=f36cc14a79404ebbaa216562f0f34fa1)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:38.821386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:38.845958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:38.847091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:38.847559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:38.848070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:38.849523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:38.850041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:38.850481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 300/606] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:49:38 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/a267b6f64f5f49b8b3f5837e701bc216/roles/f36cc14a79404ebbaa216562f0f34fa1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:38.852385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:38.852649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:38.887896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 307/607] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:38.894747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:38.895744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:39.083756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:39.092280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e75c5-cc4b-4cab-929f-db06f664adb8 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:39.144854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94fcb2-21fe-4d7d-ac63-2c68fc7c26bf None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:39.149556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:39.151810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.152907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.153428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.154427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:39.155019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:39.156672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:39.179825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 301/608] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:39.187309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:39.216459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:39.217841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.218309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.218779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.240137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:39.243414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:39.278719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:39.279756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:39.280635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:39.280635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:39.280635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:39.280635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:39.282429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 308/609] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:39.290367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:39.409237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:39.454948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0430-abdb-4c06-a761-ccdd1251d398 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:39.486044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8685c0-5362-4a5a-95a5-3de4bf6dc496 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:39.514311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:39.527653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeefa56-d746-4b6f-93cd-c4d2b4448685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:39.528291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeefa56-d746-4b6f-93cd-c4d2b4448685 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.529229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeefa56-d746-4b6f-93cd-c4d2b4448685 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.529611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeefa56-d746-4b6f-93cd-c4d2b4448685 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.530242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955a44f-201b-470c-aa89-ce8d50fd06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:39.530556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeefa56-d746-4b6f-93cd-c4d2b4448685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:39.530962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeefa56-d746-4b6f-93cd-c4d2b4448685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:39.532223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a44f-201b-470c-aa89-ce8d50fd06b0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.532570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeefa56-d746-4b6f-93cd-c4d2b4448685 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:39.533193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a44f-201b-470c-aa89-ce8d50fd06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.533861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a44f-201b-470c-aa89-ce8d50fd06b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.548777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955a44f-201b-470c-aa89-ce8d50fd06b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:39.551995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955a44f-201b-470c-aa89-ce8d50fd06b0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:39.553283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 309/610] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:39.555612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f62ef3-dbaf-4cb9-aea9-5701bc3da87e None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.556194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f62ef3-dbaf-4cb9-aea9-5701bc3da87e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.556353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f62ef3-dbaf-4cb9-aea9-5701bc3da87e None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.558319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 310/611] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:39.559273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f62ef3-dbaf-4cb9-aea9-5701bc3da87e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c955a44f-201b-470c-aa89-ce8d50fd06b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:39.586689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:39.588358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:39.590380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:39.594809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 302/612] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:39.597898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca36c-8e13-4337-9e62-0d8be621955d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.598862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca36c-8e13-4337-9e62-0d8be621955d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.599710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca36c-8e13-4337-9e62-0d8be621955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.632696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f03ca36c-8e13-4337-9e62-0d8be621955d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:39.634877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f03ca36c-8e13-4337-9e62-0d8be621955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:39.693738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f62ef3-dbaf-4cb9-aea9-5701bc3da87e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:39.720519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 303/613] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:39.721839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ca36c-8e13-4337-9e62-0d8be621955d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:39.733785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:39.734968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.735091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.735464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.745191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:39.746831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:39.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:39.764934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:39.766109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:39.771322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 311/614] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:39.773565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:39.813503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:39.851800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ca36c-8e13-4337-9e62-0d8be621955d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:39.857053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:39.935722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:39.936825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3d478-5736-4bd1-b1a9-bee0241ca440 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:39.936940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.937450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.937965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.939761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:39.941045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:39.942807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:39.972219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 304/615] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:39.974793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:39.989164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:39.989854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:39.990135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:39.990443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:39.991397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:39.991714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:39.993219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:40.018158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 312/616] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jun 29 07:49:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:40.019360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:40.167778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5ec07-992c-48c5-bdc5-8f3756c2d4f5 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:40.180756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92deea9a-3b33-4536-a45b-8c801630a01d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:40.184661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:40.195854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:40.209607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:40.211097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:40.211891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:40.212549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:40.225326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:40.234161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:40.235863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:40.237553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:40.240716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 305/617] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:40.242000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:40.263192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:40.265510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:40.266329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:40.267182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/45a558f9c7c84a05a90595fda7fccd40/roles/f36cc14a79404ebbaa216562f0f34fa1` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:40.299272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=45a558f9c7c84a05a90595fda7fccd40, role_id=f36cc14a79404ebbaa216562f0f34fa1)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:40.302982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:40.341021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 313/618] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:49:40 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/45a558f9c7c84a05a90595fda7fccd40/roles/f36cc14a79404ebbaa216562f0f34fa1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:40.341021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:40.410188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bf6be-8174-4259-9229-2abcd69e99ce None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:40.523321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:40.524315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:40.524607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:40.524889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:40.550230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:40.551888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:40.606146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab58d5d-3a96-4a99-b261-3d32a7a8da3f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:40.612269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:40.613861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:40.615537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:40.615537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:40.615537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:40.615537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:40.619716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 306/619] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:40.627337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:40.705389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:40.708265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:40.715250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:40.716151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:40.716906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:40.718663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:40.728801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:40.731708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:40.755156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 314/620] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:40.761427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:40.815527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:40.873881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc68f6-0a29-42ad-8b66-a5dd8d18b6f2 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:40.953601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:40.955312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:40.962488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:40.963434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:40.965184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:40.966036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:40.968957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:41.010012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 307/621] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:41.018345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:41.049002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394c2da-119e-4986-b6ae-483ae1dfe790 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:41.091056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:41.092218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.092725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.093246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.107412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:41.109329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:41.140635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:41.141687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:41.142559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:41.142559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:41.142559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:41.142559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:41.145396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 315/622] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:41.152154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:41.314721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:41.333122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80b712-2997-49d9-a4e8-2cce2e4e344e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:41.333395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:41.402260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24814e3-38ec-4f4c-8922-a594ce49d15f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:41.430220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:41.431114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.431406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.431686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.450385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:41.451878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:41.474444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:41.476196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.476947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.477544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.478492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:41.479002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:41.480610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:41.500938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:41.502186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:41.503076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:41.506509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 308/623] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 488 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:41.513742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:41.516802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 316/624] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:41.522019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:41.654659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea986b82-8222-4ea7-9b0d-5bb4555a0131 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:41.712799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b4cc6-1a30-4226-8528-65a9fc4dd751 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:41.715279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3324e0c-320a-4176-b10a-109fca03a18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:41.716399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3324e0c-320a-4176-b10a-109fca03a18f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.716881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3324e0c-320a-4176-b10a-109fca03a18f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.717414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3324e0c-320a-4176-b10a-109fca03a18f None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.718893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3324e0c-320a-4176-b10a-109fca03a18f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:41.719442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3324e0c-320a-4176-b10a-109fca03a18f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:41.722195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3324e0c-320a-4176-b10a-109fca03a18f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:41.745445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:41.746981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 309/625] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:41.755427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99eeab-7db1-428b-8cc1-f9c146da2d5d None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.755966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99eeab-7db1-428b-8cc1-f9c146da2d5d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.756423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99eeab-7db1-428b-8cc1-f9c146da2d5d None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.757827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e3dbb-ab7c-4d61-a2b3-79474e8c619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:41.758910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e3dbb-ab7c-4d61-a2b3-79474e8c619e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.759378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e3dbb-ab7c-4d61-a2b3-79474e8c619e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.759820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e3dbb-ab7c-4d61-a2b3-79474e8c619e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.760433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 310/626] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:41.762033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99eeab-7db1-428b-8cc1-f9c146da2d5d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:41.774256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e3dbb-ab7c-4d61-a2b3-79474e8c619e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:41.777116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e3dbb-ab7c-4d61-a2b3-79474e8c619e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:41.793326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-e69e3dbb-ab7c-4d61-a2b3-79474e8c619e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:41.802925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:41.803996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:49:41.805106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:41.805730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 317/627] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:41.808146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e8934-910e-475f-a826-38a356faa7b7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.808632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e8934-910e-475f-a826-38a356faa7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.809278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e8934-910e-475f-a826-38a356faa7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.841756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-726e8934-910e-475f-a826-38a356faa7b7 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:41.843809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-726e8934-910e-475f-a826-38a356faa7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:41.916594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99eeab-7db1-428b-8cc1-f9c146da2d5d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:41.961342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 318/628] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:41.962445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e8934-910e-475f-a826-38a356faa7b7 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:41.963832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:41.965238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:41.965737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:41.966239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/a411a835bce0475fae1db6e099dc2605/roles/f36cc14a79404ebbaa216562f0f34fa1` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:41.985743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=a411a835bce0475fae1db6e099dc2605, role_id=f36cc14a79404ebbaa216562f0f34fa1)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:41.987744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:42.007190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 311/629] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:49:41 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/a411a835bce0475fae1db6e099dc2605/roles/f36cc14a79404ebbaa216562f0f34fa1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:49:42.009459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:42.149852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:42.153332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e8934-910e-475f-a826-38a356faa7b7 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:42.197731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:42.217299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c194c1-5eb6-4156-867e-30ace02d667a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:42.222615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:42.223314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:42.223603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:42.223930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:42.224794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:42.225132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:42.227192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:42.262171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:42.264931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:42.266200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:42.266895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:42.268280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 319/630] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jun 29 07:49:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:42.273774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:42.292207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:42.312282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:42.314259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:42.316691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:42.322156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 312/631] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:42.330588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:42.476103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a159a2-9823-412c-b365-0df43824efb0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:42.518033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa26b6-0478-4974-8fbc-9ac9a4cf5227 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:42.556133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca630a-b4e9-41cf-8a67-82ddd8765a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:42.556996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca630a-b4e9-41cf-8a67-82ddd8765a92 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:42.557486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca630a-b4e9-41cf-8a67-82ddd8765a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:42.557957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca630a-b4e9-41cf-8a67-82ddd8765a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:42.563412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:42.582819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f046e9-c928-41b8-b3d3-15bba0ed9274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:42.584217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f046e9-c928-41b8-b3d3-15bba0ed9274 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:42.584831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f046e9-c928-41b8-b3d3-15bba0ed9274 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:42.585981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f046e9-c928-41b8-b3d3-15bba0ed9274 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:42.587000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f046e9-c928-41b8-b3d3-15bba0ed9274 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:42.587694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f046e9-c928-41b8-b3d3-15bba0ed9274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:42.590415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f046e9-c928-41b8-b3d3-15bba0ed9274 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:42.613219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 313/632] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:42.616002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed41a-56ca-44e9-ad96-272d21e00c7d None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:42.617790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed41a-56ca-44e9-ad96-272d21e00c7d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:42.618271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed41a-56ca-44e9-ad96-272d21e00c7d None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:42.627046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 314/633] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:42.627046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed41a-56ca-44e9-ad96-272d21e00c7d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:42.652729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:42.704294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca630a-b4e9-41cf-8a67-82ddd8765a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:42.705823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca630a-b4e9-41cf-8a67-82ddd8765a92 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:42.790923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed41a-56ca-44e9-ad96-272d21e00c7d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:42.823162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 320/634] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:49:42.824627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b09262-e718-4113-ac87-976c16b41bff None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:42.824919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b09262-e718-4113-ac87-976c16b41bff None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:42.825220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b09262-e718-4113-ac87-976c16b41bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:42.832040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:42.833238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:42.833728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:42.834222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:42.842644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-24b09262-e718-4113-ac87-976c16b41bff None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:42.844188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-24b09262-e718-4113-ac87-976c16b41bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:42.846454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:42.848147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:42.868223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:42.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:42.870690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:42.870690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:42.870690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:42.870690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:42.873341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 315/635] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:42.874609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:42.923078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:42.961012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 321/636] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:42.961012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b09262-e718-4113-ac87-976c16b41bff None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:42.961012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:43.031143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd0838-25dd-4b19-8c92-902372a8dfe6 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.078457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.079720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.080213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.080727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.087287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b09262-e718-4113-ac87-976c16b41bff None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.095598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:43.097591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:43.121252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:43.122268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:43.123176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:43.123176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:43.123176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:43.123176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:43.125633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 316/637] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:43.126717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:43.128690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.129400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.129683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.130277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.131250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:43.131560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:43.134114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:43.150124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 322/638] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:43.151078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:43.228539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:43.258172 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:43.269970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf1bb3-b2bc-41e1-aee1-42c7942aa91d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.305552 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9bb72-fad1-4a23-9a38-174bbd68ef3d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.334142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb5dc3-e580-4ba9-9254-0793a474db09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.335684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb5dc3-e580-4ba9-9254-0793a474db09 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.336046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb5dc3-e580-4ba9-9254-0793a474db09 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.336484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb5dc3-e580-4ba9-9254-0793a474db09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.356148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb5dc3-e580-4ba9-9254-0793a474db09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-2ceb5dc3-e580-4ba9-9254-0793a474db09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:43.368528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:43.369981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:43.371678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:43.372971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.372971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.372971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.372971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.373498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:43.373498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:43.375226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:43.378744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 317/639] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:43.380855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73761c6c-194e-4985-811a-58a70ba36329 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.381351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73761c6c-194e-4985-811a-58a70ba36329 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.381881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73761c6c-194e-4985-811a-58a70ba36329 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.389066 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 318/640] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:43.389066 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73761c6c-194e-4985-811a-58a70ba36329 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:43.409601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 323/641] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:43.410960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:43.546441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73761c6c-194e-4985-811a-58a70ba36329 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.568580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff339-4755-4b03-a71c-b6c06a9c56b9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.601613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e440186-7b38-4f8d-91d1-4ba45fc287d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.602493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e440186-7b38-4f8d-91d1-4ba45fc287d1 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.602878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e440186-7b38-4f8d-91d1-4ba45fc287d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.603305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e440186-7b38-4f8d-91d1-4ba45fc287d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.613766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e440186-7b38-4f8d-91d1-4ba45fc287d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:43.614515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:43.615231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e440186-7b38-4f8d-91d1-4ba45fc287d1 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:43.635715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.636952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-1e440186-7b38-4f8d-91d1-4ba45fc287d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:43.637244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:43.639051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 07:49:43.641432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:43.643967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.643967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.643967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:43.643967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:43.643967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:43.645386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 319/642] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:43.645386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c3265-ffce-40f4-8f8d-0400a54d3627 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.645386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c3265-ffce-40f4-8f8d-0400a54d3627 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.645386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c3265-ffce-40f4-8f8d-0400a54d3627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.659852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 324/643] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:43.666908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:43.667557 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a07c3265-ffce-40f4-8f8d-0400a54d3627 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:43.668781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a07c3265-ffce-40f4-8f8d-0400a54d3627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:43.672131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:43.775167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 320/644] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:43.778424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c3265-ffce-40f4-8f8d-0400a54d3627 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:43.825986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43723dd5-486a-4d54-b4f0-0619bacc9131 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.889213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.891004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.891565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.892023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.904721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c3265-ffce-40f4-8f8d-0400a54d3627 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:43.905342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:43.907813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:43.934888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:43.936625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 325/645] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:43.938602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:43.955353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:43.956042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:43.956326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:43.956607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:43.957508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:43.957816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:43.959293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:43.971549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:43.989255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 321/646] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:43.991967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:44.021659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:44.124678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d46d7d-1013-4be5-97a9-2824576de2d2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:44.156306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b020d-f7a7-47d1-b750-10ffd66d9c5c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:44.174271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:44.175151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.175347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.175609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.189982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:44.191461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:44.213881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:44.215575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:44.217273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:44.217273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:44.217273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:44.217273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:44.217273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 326/647] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:44.217273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:44.230748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:44.232130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.232634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.233195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.252926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:44.268057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:44.270134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 322/648] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:44.271983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:44.361220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:44.370551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:44.374856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50ba3-45ca-409b-b335-135ea9dd3036 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:44.417738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085f494-49fb-47d4-aad1-8a6032cee441 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:44.418686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:44.419365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.419651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.419933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.420747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:44.421091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:44.422576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:44.436599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 327/649] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:44.438153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:44.461563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80398c1-eb7f-4b9a-8982-6b205cddc5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:44.462406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80398c1-eb7f-4b9a-8982-6b205cddc5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.462835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80398c1-eb7f-4b9a-8982-6b205cddc5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.463293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80398c1-eb7f-4b9a-8982-6b205cddc5b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.547989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80398c1-eb7f-4b9a-8982-6b205cddc5b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:44.549785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80398c1-eb7f-4b9a-8982-6b205cddc5b5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:44.557945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61499ee6-8338-4e92-9afa-fb98469e1c01 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:44.595904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:44.596962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.597427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.597795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.598618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:44.599015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:44.601018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:44.613001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:44.617099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 328/650] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:44.618421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:44.649193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:44.689153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 323/651] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:44.690819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e2e3-f79f-4211-989f-987f763eda04 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.691256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e2e3-f79f-4211-989f-987f763eda04 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.691675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e2e3-f79f-4211-989f-987f763eda04 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.697418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 324/652] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:44.699281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e2e3-f79f-4211-989f-987f763eda04 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:44.753506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e79ae-ab8c-41ba-9ccc-9f0309a2350a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:44.813976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:44.815117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.815597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.816067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.816941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:44.817282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:44.819975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:44.841408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 329/653] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:44.842753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:44.855734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e2e3-f79f-4211-989f-987f763eda04 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:44.900718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8846479-3707-447e-b35e-21872e56e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:44.901799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846479-3707-447e-b35e-21872e56e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.902287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846479-3707-447e-b35e-21872e56e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.902783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846479-3707-447e-b35e-21872e56e0c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.903963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8846479-3707-447e-b35e-21872e56e0c0 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=24630ccd20794f19a138a19187348920) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:44.904504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8846479-3707-447e-b35e-21872e56e0c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:44.906906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8846479-3707-447e-b35e-21872e56e0c0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:44.915422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 325/654] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:44 2022] GET /identity/v3/role_assignments?user.id=24630ccd20794f19a138a19187348920&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:44.917094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997801a6-4824-4b59-a94d-970083dceabf None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:44.917607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997801a6-4824-4b59-a94d-970083dceabf None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:44.918109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997801a6-4824-4b59-a94d-970083dceabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:44.948401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-997801a6-4824-4b59-a94d-970083dceabf None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:44.950654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-997801a6-4824-4b59-a94d-970083dceabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:44.975569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:45.005551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:45.057540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6e38-1b53-4ae3-8825-f2db9595bbf9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:45.058892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 326/655] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:45.061008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997801a6-4824-4b59-a94d-970083dceabf None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:45.151641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:45.153444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:45.155383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:45.155889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:45.168680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:45.171206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:45.199774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:45.201236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 330/656] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:49:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:45.203794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:45.252850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997801a6-4824-4b59-a94d-970083dceabf None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:45.311790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:45.313553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:45.314266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:45.315000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:45.341200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:45.344897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:45.380615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:45.383650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:45.386609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:45.386609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:45.386609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:45.386609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:45.386609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 327/657] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:45.386609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:45.393770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:45.402694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2989db-67c7-47f1-985e-30e2b6ff1d47 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:45.455329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:45.455985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:45.456271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:45.456596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:45.469147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:45.472807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:45.474353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:45.495370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:45.496625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:45.498424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:45.498424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:45.498424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:45.498424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:45.498424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 331/658] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:45.498424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:45.571077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765fa52-b93b-431e-9822-2395fe03c669 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:45.620199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7992-1204-43e6-b4e5-be13e6726619 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:45.627903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:45.628734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:45.629062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:45.629351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:45.642307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:45.650957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:45.653077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 328/659] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:45.654639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:45.685887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:45.686613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:45.686926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:45.687211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:45.688097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:45.688420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:45.690008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:45.716492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 332/660] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:45.716492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:45.766743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:45.776666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:45.830342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be356-ab3b-4284-9c7b-e06c478e3423 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:45.843457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cdaae-f098-4353-bca1-8697986ad4a0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:45.877627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:45.878326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:45.878616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:45.878898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:45.879655 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:45.879972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:45.881409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:45.900297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:45.901007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:45.901397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:45.901633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:45.902309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:45.902618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:45.903529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 329/661] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:45.904240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:45.909308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:45.929012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 333/662] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:45.929012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:46.043292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ada7b5-45d9-4759-a2f0-d1b4d38f40d4 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.055003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:46.081937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a9098-1c94-4ee1-a2fe-b566a9db2584 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.082927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:46.083655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:46.083964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:46.084277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:46.127235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:46.146829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:46.148072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:46.148825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:46.149371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:46.150823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:46.151338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:46.153935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:46.168118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:46.169625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:46.175604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 330/663] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:46.175604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:46.298818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 334/664] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:46.298818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:46.310178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:46.351299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87733619-4ed8-4f0f-84a8-fd971cb49280 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.391247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:46.392054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:46.392336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:46.392608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:46.401346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:46.402824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:46.417930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:46.418914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:46.419916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:46.424130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 331/665] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:46.425089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:46.456607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55cc4-7c5f-4c43-b406-2149514ded29 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.497158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:46.504521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:46.505614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:46.506049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:46.506476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:46.517583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:46.519585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:46.537393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:46.538506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:46.539457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 335/666] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:46.540065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:46.593128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73930277-dc54-4402-b74a-3013d3ae22d6 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.624223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:46.649323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:46.650150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:46.650467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:46.650743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:46.665949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912bb1b-1748-4e75-9bf3-b0f8e6de48bd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.666774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:46.669411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:46.703521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:46.704512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:46.705479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:46.705479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:46.705479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:46.705479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:46.705721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 332/667] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:46.706996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:46.720230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:46.722251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:46.723149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:46.723775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:46.745968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:46.749659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:46.785709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:46.787594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:46.790179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:46.790179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:46.790179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:46.790179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:46.793296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 336/668] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:46.795071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:46.897410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:46.906913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37834646-1293-43d2-a4aa-d6dac09a11cf None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.962764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:46.963673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:46.964113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:46.964545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:46.965577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:46.966122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:46.968495 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:46.987665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 333/669] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:46.988750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbac5c0-0278-42ee-bd93-12d70befb082 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:46.989287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:47.031503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:47.051875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:47.053143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.053795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.054758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:47.056306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:47.056937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:47.059309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:47.094959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 337/670] 10.222.0.43 () {56 vars in 1139 bytes} [Wed Jun 29 07:49:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:47.094959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:47.161475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6021521-9062-49a0-9ce7-b26c5eaf81b7 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:47.216192 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:47.217295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.217753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.218208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:47.219485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:47.219968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:47.222129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:47.248079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 334/671] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:47.248079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:47.258329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de897f2a-4567-4398-9d60-9f518a875cdd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:47.275621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:47.313799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:47.315316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.315794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.316269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:47.317566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:47.318482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:47.321132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:47.340865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 338/672] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:47.342163 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:47.409591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:47.422474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653bc5e-06b7-4642-becb-60457cc26090 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:47.461999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:47.462682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.463192 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.463470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:47.464326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=30d3144975804e369a023316655be2d0) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:47.464637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:47.466142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:47.472626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 335/673] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:47 2022] GET /identity/v3/role_assignments?user.id=30d3144975804e369a023316655be2d0&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:47.473507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:47.508548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc37f-c580-46c5-94e8-e26b1538ce3e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:47.578025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd774642-a9c0-4a42-a4bc-0102c7f92d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:47.579218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd774642-a9c0-4a42-a4bc-0102c7f92d8f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.579699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd774642-a9c0-4a42-a4bc-0102c7f92d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.580282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd774642-a9c0-4a42-a4bc-0102c7f92d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:47.608763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:47.638396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3556d7-15e3-469e-ac5f-d82bbd12d255 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:47.709200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd774642-a9c0-4a42-a4bc-0102c7f92d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:47.711218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd774642-a9c0-4a42-a4bc-0102c7f92d8f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:47.714043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:47.714897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.715184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.715464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:47.727160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:47.728721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:47.748030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:47.749249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 336/674] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:47.750185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:47.795582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:47.912786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 339/675] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:47.915128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155b00f-1bdb-4c40-b8f1-91adb7febee3 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.915648 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155b00f-1bdb-4c40-b8f1-91adb7febee3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.916138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155b00f-1bdb-4c40-b8f1-91adb7febee3 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:47.919191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 340/676] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:47.920720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155b00f-1bdb-4c40-b8f1-91adb7febee3 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:47.937915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb57ca-1d4e-42ce-b5a6-d7d9dd00bbca None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:47.997806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f220e-71b3-4f47-8bc1-4202c5b02f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:47.998729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f220e-71b3-4f47-8bc1-4202c5b02f8a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:47.999141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f220e-71b3-4f47-8bc1-4202c5b02f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:47.999575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f220e-71b3-4f47-8bc1-4202c5b02f8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.009174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f220e-71b3-4f47-8bc1-4202c5b02f8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:48.011363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f220e-71b3-4f47-8bc1-4202c5b02f8a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-408f220e-71b3-4f47-8bc1-4202c5b02f8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:48.029382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:48.031122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:48.032905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 337/677] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:48.035652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ef51f-8d7e-4320-bde5-683a4241d238 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:48.035652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ef51f-8d7e-4320-bde5-683a4241d238 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:48.035652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ef51f-8d7e-4320-bde5-683a4241d238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.043837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:48.062563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-db2ef51f-8d7e-4320-bde5-683a4241d238 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:48.064024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-db2ef51f-8d7e-4320-bde5-683a4241d238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:48.152911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155b00f-1bdb-4c40-b8f1-91adb7febee3 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:48.168377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 338/678] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:48.169544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ef51f-8d7e-4320-bde5-683a4241d238 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:48.216683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:48.218084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:48.218706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:48.219309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.221126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:48.221839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:48.224677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:48.244565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:48.273457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 341/679] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jun 29 07:49:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:48.273457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:48.335121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ef51f-8d7e-4320-bde5-683a4241d238 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:48.375182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:48.381363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:48.383067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:48.383494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:48.384039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.400851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:48.402364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:48.422885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:48.425620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:48.427370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:48.427370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:48.427370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:48.427370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:48.429469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 339/680] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:48.430605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:48.436768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cd160-b6ea-4338-8bea-cae8c471c440 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:48.501934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:48.503062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:48.503554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:48.504006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.505277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:48.505785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:48.508170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:48.535185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 342/681] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:48.535185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:48.639706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ce075-4eef-45ef-a920-b5f9e3519902 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:48.640631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:48.703910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:48.705366 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:48.705882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:48.706367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.706617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41edf12-030b-4e4c-8c45-d2a7491f8492 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:48.707558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:48.710550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:48.738134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:48.739285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:49:48.740119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:48.742466 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 340/682] 10.222.0.43 () {56 vars in 1129 bytes} [Wed Jun 29 07:49:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:48.742918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:48.777540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:48.778580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:48.778995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:48.779579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.832890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:48.883276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:48.884798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:48.909131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c130d2-2d1e-41e4-aefb-ce081457181b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:48.950948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:48.951753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:48.952048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:48.952322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:48.953376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:48.953549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:48.955196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:48.962620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:48.979885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 341/683] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:48.979885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.030669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 343/684] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:49.036253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.109476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292c5be-acc7-4bbe-98ee-810eec0c6305 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:49.122473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:49.161659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:49.162325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:49.163179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:49.163464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:49.164311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a267b6f64f5f49b8b3f5837e701bc216) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:49.164629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:49.166129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:49.172306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26d375-bf33-4a44-a6ff-c93350776d48 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:49.173148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 342/685] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:48 2022] GET /identity/v3/role_assignments?user.id=a267b6f64f5f49b8b3f5837e701bc216&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:49.173816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.237151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:49.238662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:49.239187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:49.239668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:49.240608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:49.241929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:49.243942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:49.257595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 344/686] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jun 29 07:49:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:49.257595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.293207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:49.327706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adea74-0d80-4c6c-8f83-9dede7c9fa82 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:49.380429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:49.381725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:49.382897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:49.383454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:49.393572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:49.396228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:49.399582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c4741-c07b-42d5-ac3c-dd72a9513efc None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:49.426206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:49.427486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:49.428903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:49.429950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 343/687] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:49.431462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.446422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:49.447502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:49.447789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:49.448698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:49.463369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:49.464957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:49.491995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:49.493276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:49.494989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:49.496834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:49.496834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:49.496834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:49.496834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:49.496834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 345/688] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:49.496834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.623197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176daa93-7676-410b-89e5-8464e218ba00 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:49.643421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:49.674693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fc1e9-e849-4dba-a707-ae6b6a619a31 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:49.689113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:49.690501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:49.691226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:49.692085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:49.693546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:49.694391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:49.697260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:49.706540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 344/689] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jun 29 07:49:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:49.708136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.741651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:49.742943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:49.743524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:49.744050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:49.745683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:49.746256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:49.755627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:49.797041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 346/690] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:49.804724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:49.919992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:49.930941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9005a-457e-430d-bff5-b4ee1d5a5668 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:49.992002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:49.993294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:49.993764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:49.994191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:49.995090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:49.995559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:49.997143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:50.016324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 345/691] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:50.023527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:50.044469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231720f1-986b-4d69-ace9-4b1eee6d574d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:50.121314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52128002-3184-4b1e-a765-04e57a9014c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:50.122238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128002-3184-4b1e-a765-04e57a9014c6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.122774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128002-3184-4b1e-a765-04e57a9014c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.123198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128002-3184-4b1e-a765-04e57a9014c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.124191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128002-3184-4b1e-a765-04e57a9014c6 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=45a558f9c7c84a05a90595fda7fccd40) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:50.124654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128002-3184-4b1e-a765-04e57a9014c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:50.126296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128002-3184-4b1e-a765-04e57a9014c6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:50.128543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:50.136359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 347/692] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:49 2022] GET /identity/v3/role_assignments?user.id=45a558f9c7c84a05a90595fda7fccd40&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:50.224377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3f555-f671-4ebf-bb4f-6cf6e289bacb None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:50.246789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39022b19-96cc-4a49-81f9-b3c1a4377339 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.247354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39022b19-96cc-4a49-81f9-b3c1a4377339 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.247783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39022b19-96cc-4a49-81f9-b3c1a4377339 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.249854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 348/693] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:49:50.258972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149c96-83c5-4579-8790-a36783a72f02 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.259634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149c96-83c5-4579-8790-a36783a72f02 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.260125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149c96-83c5-4579-8790-a36783a72f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.289050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d9149c96-83c5-4579-8790-a36783a72f02 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:50.291898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d9149c96-83c5-4579-8790-a36783a72f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:50.300434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:50.301771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.302266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.302801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.321126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:50.325589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:50.355539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:50.356525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:50.357453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:50.364194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 346/694] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:50.371703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:50.411191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:50.468207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 349/695] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:50.493918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149c96-83c5-4579-8790-a36783a72f02 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:50.546641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c36e23-c841-487e-9fa8-86a0287cff88 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:50.593380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:50.594215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.594327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.594592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.595396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:50.595709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:50.597337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:50.612293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 347/696] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:50.621568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:50.631207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149c96-83c5-4579-8790-a36783a72f02 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:50.665827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:50.676565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:50.677575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.677960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.678451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.696639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:50.698174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:50.719834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:50.721675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:50.723506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:50.723506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:50.723506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:50.723506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:50.723506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 350/697] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:50.734597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:50.808263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:50.866899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf99f3-87a5-4d2d-9e9d-8ebba4326573 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:50.886794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4d72e-8472-4632-b172-50e0940660af None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:50.927213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:50.927901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.928187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.928187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.930275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:50.930595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:50.932198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:50.951502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 351/698] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:50.955451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd02b6-d2ef-48ab-89f9-8f6ca246ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:50.956122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd02b6-d2ef-48ab-89f9-8f6ca246ec1d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:50.956409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd02b6-d2ef-48ab-89f9-8f6ca246ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:50.956692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd02b6-d2ef-48ab-89f9-8f6ca246ec1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:50.957579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd02b6-d2ef-48ab-89f9-8f6ca246ec1d None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a411a835bce0475fae1db6e099dc2605) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:50.957893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd02b6-d2ef-48ab-89f9-8f6ca246ec1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:50.960494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:50.962303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd02b6-d2ef-48ab-89f9-8f6ca246ec1d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:50.983100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 348/699] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:50 2022] GET /identity/v3/role_assignments?user.id=a411a835bce0475fae1db6e099dc2605&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:51.033611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:51.102283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1968-180b-441b-a4ee-c69649cd9a02 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:51.146173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914b21b-6f9a-4bfd-b2ca-cccae96d30aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:51.146965 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914b21b-6f9a-4bfd-b2ca-cccae96d30aa None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:51.147945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914b21b-6f9a-4bfd-b2ca-cccae96d30aa None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:51.148433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914b21b-6f9a-4bfd-b2ca-cccae96d30aa None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:51.149357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914b21b-6f9a-4bfd-b2ca-cccae96d30aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:51.150853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914b21b-6f9a-4bfd-b2ca-cccae96d30aa None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:51.186205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 352/700] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:49:50 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:49:52.419735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad963bdf-be4e-49e7-9ea6-ae5342e674b3 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:52.420707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad963bdf-be4e-49e7-9ea6-ae5342e674b3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:52.420707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad963bdf-be4e-49e7-9ea6-ae5342e674b3 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:52.423792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 349/701] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:49:52.430955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c29e5c-a27d-4b8b-b9bb-47396dd73086 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:52.431785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c29e5c-a27d-4b8b-b9bb-47396dd73086 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:52.431942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c29e5c-a27d-4b8b-b9bb-47396dd73086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:52.468884 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-74c29e5c-a27d-4b8b-b9bb-47396dd73086 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:52.470932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-74c29e5c-a27d-4b8b-b9bb-47396dd73086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:52.604875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 353/702] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:52.628743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad963bdf-be4e-49e7-9ea6-ae5342e674b3 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:52.747397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd8ec8-886f-463a-98d9-0b5d3f565037 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:52.748291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd8ec8-886f-463a-98d9-0b5d3f565037 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:52.748967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd8ec8-886f-463a-98d9-0b5d3f565037 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:52.751958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 354/703] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:49:52.761908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f591f-2fd9-4cb7-ae49-a18467c7a774 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:52.762830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f591f-2fd9-4cb7-ae49-a18467c7a774 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:52.763591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f591f-2fd9-4cb7-ae49-a18467c7a774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:52.816642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3d1f591f-2fd9-4cb7-ae49-a18467c7a774 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:52.819665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3d1f591f-2fd9-4cb7-ae49-a18467c7a774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:52.863424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad963bdf-be4e-49e7-9ea6-ae5342e674b3 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:52.895784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:52.944745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:52.946971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:52.947280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:52.947538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:52.974806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:52.989925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:52.999920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:53.001165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:53.002074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:53.002850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 350/704] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:53.011260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:53.035344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 355/705] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 07:49:53.138616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f591f-2fd9-4cb7-ae49-a18467c7a774 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:53.243538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab217b0-be3a-4133-a90d-3dad019ffc70 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:53.324489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447478fe-5eeb-4324-96ef-45d570d8d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:53.325621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447478fe-5eeb-4324-96ef-45d570d8d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:53.326108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447478fe-5eeb-4324-96ef-45d570d8d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:53.326608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447478fe-5eeb-4324-96ef-45d570d8d5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:53.328002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447478fe-5eeb-4324-96ef-45d570d8d5d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:53.328533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447478fe-5eeb-4324-96ef-45d570d8d5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:53.331442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447478fe-5eeb-4324-96ef-45d570d8d5d2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:53.349364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 351/706] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:53.352581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c742f10-7b9d-4eb6-98e6-7d1a559bbae1 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:53.353084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c742f10-7b9d-4eb6-98e6-7d1a559bbae1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:53.353707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c742f10-7b9d-4eb6-98e6-7d1a559bbae1 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:53.356754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 352/707] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:53.359594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c742f10-7b9d-4eb6-98e6-7d1a559bbae1 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:53.387461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f591f-2fd9-4cb7-ae49-a18467c7a774 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:53.458846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:53.465591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9f5ad-3331-4b41-b188-d415bad7d8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:53.467684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f5ad-3331-4b41-b188-d415bad7d8f3 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:53.468464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f5ad-3331-4b41-b188-d415bad7d8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:53.469240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f5ad-3331-4b41-b188-d415bad7d8f3 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:53.470920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9f5ad-3331-4b41-b188-d415bad7d8f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:53.473946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9f5ad-3331-4b41-b188-d415bad7d8f3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:53.509917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 356/708] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:49:53 2022] POST /identity/v3/services => generated 251 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:49:53.512872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99e053-bc7f-4dc6-bc7c-4013e837364e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:53.513757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99e053-bc7f-4dc6-bc7c-4013e837364e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:53.514500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99e053-bc7f-4dc6-bc7c-4013e837364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:53.545387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ad99e053-bc7f-4dc6-bc7c-4013e837364e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:53.548309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ad99e053-bc7f-4dc6-bc7c-4013e837364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:53.574773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:53.604927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c742f10-7b9d-4eb6-98e6-7d1a559bbae1 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:53.694731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 357/709] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:53.731718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99e053-bc7f-4dc6-bc7c-4013e837364e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:53.739838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:53.742692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:53.743171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:53.743665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:53.783862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:53.791091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:53.867390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:53.869065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:53.870770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:53.870770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:53.870770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:53.870770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:53.874651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 353/710] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 516 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:53.882115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:54.035841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99e053-bc7f-4dc6-bc7c-4013e837364e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:54.115216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:54.120206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.120645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.121193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] PATH_INFO: `/v3/services/network` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.122472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:54.126832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:54.133573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae1e68-00f0-4309-881b-f85e83e904df None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:54.157254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:54.157530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:54.159378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:54.162100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 358/711] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:53 2022] GET /identity/v3/services/network => generated 88 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:49:54.169928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:54.193026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:54.221619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:54.223404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.224224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.225558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.227341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:54.228206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:54.231282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:54.258931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 354/712] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:54.265401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:54.374879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0fe9-630e-4e9d-bfda-c52b61c07745 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:54.448572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcc9cd-5031-41b6-8184-8ec1997b2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:54.451035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcc9cd-5031-41b6-8184-8ec1997b2fcc None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.451894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcc9cd-5031-41b6-8184-8ec1997b2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.452820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcc9cd-5031-41b6-8184-8ec1997b2fcc None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.454600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcc9cd-5031-41b6-8184-8ec1997b2fcc None admin] RBAC: Adding query filter params (name=network) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:54.455474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcc9cd-5031-41b6-8184-8ec1997b2fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:54.458656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcc9cd-5031-41b6-8184-8ec1997b2fcc None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:54.468167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 359/713] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:49:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:54.469263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713325-c977-4349-9fc3-47fff7e78e5f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:54.472792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c0fc-8f7b-413b-910d-5cbd37b733b7 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.473614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c0fc-8f7b-413b-910d-5cbd37b733b7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.474382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c0fc-8f7b-413b-910d-5cbd37b733b7 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.478334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 360/714] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:54.480307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5c0fc-8f7b-413b-910d-5cbd37b733b7 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:54.510059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d4475-6cad-4188-beaa-c5bc68899357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:54.510937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d4475-6cad-4188-beaa-c5bc68899357 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.511378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d4475-6cad-4188-beaa-c5bc68899357 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.511663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d4475-6cad-4188-beaa-c5bc68899357 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.520070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d4475-6cad-4188-beaa-c5bc68899357 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:54.521604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d4475-6cad-4188-beaa-c5bc68899357 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-df7d4475-6cad-4188-beaa-c5bc68899357 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:54.542590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:54.543640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:54.545103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 355/715] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:49:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:54.551644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835bb2-52cd-4067-8707-ccac11a43073 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.551974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835bb2-52cd-4067-8707-ccac11a43073 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.552262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835bb2-52cd-4067-8707-ccac11a43073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.575022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c7835bb2-52cd-4067-8707-ccac11a43073 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:54.576305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c7835bb2-52cd-4067-8707-ccac11a43073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:54.599133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:54.612422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:54.641896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5c0fc-8f7b-413b-910d-5cbd37b733b7 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:54.671929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 356/716] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:54.674291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835bb2-52cd-4067-8707-ccac11a43073 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:54.705935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:54.706615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.706900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.707177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.707914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] RBAC: Adding query filter params (type=network) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:54.708215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:54.710422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:54.718330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 361/717] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:49:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:54.719473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:54.868404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835bb2-52cd-4067-8707-ccac11a43073 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:54.944882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:54.945954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:54.946406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:54.947887 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:54.949692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:54.950239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:54.952215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:54.987978 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 357/718] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:49:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:54.999517 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a4167-fd67-463e-a532-a194c227a16e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:55.006898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:55.042251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:55.043590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.044080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.044565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.051256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:55.054623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:55.060961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:55.071701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:55.072737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:49:55.073638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:55.077216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 362/719] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:55.082990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:55.205884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbac33-7ffd-4d1a-8696-518654bca8d5 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:55.233610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7890e-9a86-409d-9011-f71cd5930129 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:55.248472 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:55.250513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.251022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.251512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.283385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d7191-1c37-4481-9218-f6594aef7c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:55.284121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7191-1c37-4481-9218-f6594aef7c16 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.284434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7191-1c37-4481-9218-f6594aef7c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.284673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7191-1c37-4481-9218-f6594aef7c16 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.285502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7191-1c37-4481-9218-f6594aef7c16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:55.286228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7191-1c37-4481-9218-f6594aef7c16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:55.289339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7191-1c37-4481-9218-f6594aef7c16 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:55.299619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 363/720] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:49:55.301364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39bbee-82c2-4385-8791-dff4df7c4dcd None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.301658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39bbee-82c2-4385-8791-dff4df7c4dcd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.301941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39bbee-82c2-4385-8791-dff4df7c4dcd None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.304219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 364/721] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:55.315318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc27817-90c7-418f-be5d-f838184c0a19 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.315628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc27817-90c7-418f-be5d-f838184c0a19 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.315889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc27817-90c7-418f-be5d-f838184c0a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.343866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-acc27817-90c7-418f-be5d-f838184c0a19 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:55.345791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-acc27817-90c7-418f-be5d-f838184c0a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:55.351865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:55.354784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:55.392004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:55.425399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:55.455063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 365/722] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:55.456596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc27817-90c7-418f-be5d-f838184c0a19 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:55.601446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 358/723] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 07:49:55.602493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:55.665872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc27817-90c7-418f-be5d-f838184c0a19 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:55.727616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750ab06-f7ff-49c9-9fb2-23dbaddb5210 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:55.735608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:55.737134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.737611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.738242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.748671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:55.761966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:55.763517 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:55.773585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:55.774411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.774697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.774973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.788106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:49:55.796507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:55.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:49:55.798643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:55.805041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:55.806771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:55.808457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:55.808457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:55.808457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:55.808457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:55.808889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 359/724] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:55.809739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 366/725] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:55.809940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:55.814193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:55.904156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:55.932908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c418e-3820-4dd7-8362-1e42915b4eee None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:55.973545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:55.974375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:55.974659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:55.975458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:55.988106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:55.989536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:56.011365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:56.013234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:56.014996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:56.014996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:56.014996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:56.014996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:56.017526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 360/726] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:56.018238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.024132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530739ea-f49c-4b62-b246-bd1e66364981 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.054558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:56.095220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.096710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.097430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.098181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.099813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:56.100650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.103627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.127692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 367/727] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:49:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:56.129694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.140899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771a9f8-e85c-46ed-bab3-808be4f3ad85 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.184307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.185016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.185325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.185606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.186452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:56.186767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.188284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.207922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 361/728] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:56.209118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.319871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:56.333073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1827347-0040-4dc8-956e-666b78692806 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.335838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e8bab-bcac-40d1-8529-90000fdfe89e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.340814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:56.374831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.375945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.376378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.376802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.389719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.391446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.394352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.395094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.395303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.395581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.396310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:56.396615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.398590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.411462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 368/729] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:56.412581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:56.418283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:56.419432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:56.420464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:56.420464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:56.420464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:56.420464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:56.423274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 362/730] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:56.424445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.535129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a3b5f-e465-4490-acb8-9a590cf5eb71 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.546729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e044e20-be2c-401f-8822-288ab1fd381a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.588960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.590131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.590598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.591072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.599267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.601473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.602309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.603143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.603636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.604134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:56.621743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:56.622734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:56.623645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:56.624094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.630758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.631729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 363/731] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:56.633341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.636627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:56.665180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:56.666637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:49:56.667603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:56.669054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 369/732] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:56.670166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.707570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:56.795084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096399c-b5f9-42ce-ad26-3518361b87ff None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.839900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.840942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.841461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.841967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.842917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:56.843431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.845070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.859924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 364/733] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:56.861196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:56.887853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c554bf-b850-48dd-b3cc-d359244cef73 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:56.933198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:56.934270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:56.934762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:56.935271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:56.936324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:56.936868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:56.938657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:56.960339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 370/734] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:56.962000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:57.092292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:57.113508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:57.115067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5032853-0df6-4e43-b36f-5ba397f97ec6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:57.163995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:57.164829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.165262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.165413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.169514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00182777-1837-4f1c-b00d-0053ed28ac0d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:57.174329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:57.175799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:49:57.193352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:57.195065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 371/735] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:57.201624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:57.219176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abecd0e-fd2a-497e-bd3d-0f612c2ceb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:57.220319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecd0e-fd2a-497e-bd3d-0f612c2ceb26 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.220818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecd0e-fd2a-497e-bd3d-0f612c2ceb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.221346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecd0e-fd2a-497e-bd3d-0f612c2ceb26 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.223003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abecd0e-fd2a-497e-bd3d-0f612c2ceb26 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:57.223555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abecd0e-fd2a-497e-bd3d-0f612c2ceb26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:57.226830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abecd0e-fd2a-497e-bd3d-0f612c2ceb26 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:57.253239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 365/736] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jun 29 07:49:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:57.254622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188409-51c1-4385-a1cb-db32fdfa9371 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.254906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188409-51c1-4385-a1cb-db32fdfa9371 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.255180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188409-51c1-4385-a1cb-db32fdfa9371 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.256999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 366/737] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:57.258120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9537e-995c-4d27-b254-21762f41ab17 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.258400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9537e-995c-4d27-b254-21762f41ab17 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.258670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9537e-995c-4d27-b254-21762f41ab17 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.260237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 367/738] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:57.261020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9537e-995c-4d27-b254-21762f41ab17 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:57.330888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1330-e207-4fd0-b43e-1bd5423e1d98 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:57.382820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:57.383850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.384347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.384837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.386511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a267b6f64f5f49b8b3f5837e701bc216) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:57.387073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:57.389884 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:57.399461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9537e-995c-4d27-b254-21762f41ab17 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:57.410763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 372/739] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:57 2022] GET /identity/v3/role_assignments?user.id=a267b6f64f5f49b8b3f5837e701bc216&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:57.410763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:57.430663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:57.433058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:57.455736 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041515c-4e40-4849-8f3b-6e8bd05cb021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:57.457361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041515c-4e40-4849-8f3b-6e8bd05cb021 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.457826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041515c-4e40-4849-8f3b-6e8bd05cb021 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.458264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041515c-4e40-4849-8f3b-6e8bd05cb021 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.459689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041515c-4e40-4849-8f3b-6e8bd05cb021 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:57.460145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041515c-4e40-4849-8f3b-6e8bd05cb021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:57.462047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041515c-4e40-4849-8f3b-6e8bd05cb021 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:57.497268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 368/740] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jun 29 07:49:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:57.499742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ba41-cc08-4e4c-9ab6-eb749087684c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.500380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ba41-cc08-4e4c-9ab6-eb749087684c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.500889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ba41-cc08-4e4c-9ab6-eb749087684c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.530649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1851ba41-cc08-4e4c-9ab6-eb749087684c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:57.532817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1851ba41-cc08-4e4c-9ab6-eb749087684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:57.582844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87caa-bbe8-4880-9851-60e4afac6cad None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:57.639255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:57.639879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.640206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.640560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.662185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 369/741] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:57.665827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6657fd-9acc-46dd-8eca-ecf63ac68716 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:57.666180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6657fd-9acc-46dd-8eca-ecf63ac68716 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:57.666649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6657fd-9acc-46dd-8eca-ecf63ac68716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:57.701338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6d6657fd-9acc-46dd-8eca-ecf63ac68716 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:57.705584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6d6657fd-9acc-46dd-8eca-ecf63ac68716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:57.755903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:57.759630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:57.780238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:57.828363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 370/742] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 07:49:57.830900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6657fd-9acc-46dd-8eca-ecf63ac68716 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:57.893163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:57.904151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 373/743] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:57.905151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.014340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6657fd-9acc-46dd-8eca-ecf63ac68716 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:58.034914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7684d4-93f9-477b-822e-456bad3015b1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:58.047555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:58.063732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:58.064404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:58.064688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:58.064963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:58.105032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:58.106563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:58.107038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:58.107498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:58.108607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:58.110972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:58.133266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:58.134638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:58.135599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 374/744] 10.222.0.43 () {56 vars in 1129 bytes} [Wed Jun 29 07:49:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:58.136760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.174007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:58.175388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:58.227021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:58.272320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8b013-7719-43f5-8dc3-33ac0dc748eb None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:58.314187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:58.314856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:58.315143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:58.315421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:58.316150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:58.317453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 371/745] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:49:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 07:49:58.317574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:58.319411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.353018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 375/746] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:49:58 2022] POST /identity/v3/services => generated 240 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:49:58.353018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.414178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:58.513903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033319c-fd70-4827-ab28-ee47c355f226 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:58.545746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef911f-1981-4419-abe1-51f5f045710b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:58.561383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:58.562595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:58.562595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:58.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:58.571798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:58.576231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:58.577375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:58.588462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:58.589206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:58.589488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:58.589772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:58.591280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:58.591280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:58.592274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:58.599958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 376/747] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jun 29 07:49:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:58.601660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:58.603515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:58.605357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:58.605357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:58.605357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:58.605357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:49:58.605357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 372/748] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:58.606790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.607516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.761337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:58.778526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9341-2b7c-432f-af33-c32afa61889f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:58.781475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbda2a6-0159-4547-b8b9-da6c3966b07b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:58.831747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:58.832804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:58.833151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:58.833539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:58.834999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:58.835372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:58.837164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:58.837486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:58.838451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:58.838948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:58.839696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:58.848970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 373/749] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jun 29 07:49:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:58.850478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.855379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:58.857273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:49:58.880476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:58.882347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:58.883379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:49:58.883379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:49:58.883379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:49:58.883379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:58.885823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 377/750] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:58.890601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:58.931001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:58.982774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da46a-5b6c-4be3-be5e-bbe28126718c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.015925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3f0b-1c97-4dc0-8cc6-2339c7a7f54e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.023348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.024476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.025046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.025555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.026579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:59.027125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.028789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.048903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 374/751] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:59.050277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.060027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.060712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.061025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.061312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.062405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] RBAC: Adding query filter params (interface=public, service_id=e4cdc5970666454baefe99b70eb734f8, region_id=RegionOne) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:59.062556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.063978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.078839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 378/752] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:49:58 2022] GET /identity/v3/endpoints?service_id=e4cdc5970666454baefe99b70eb734f8&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:59.079992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.093743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:59.180552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097eebe5-0618-4d2e-bd86-820dd52008cf None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.197695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:59.207721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd5f2e-16b0-4394-b912-f3fc9acd9a15 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.223041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.224094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.224599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.225210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.226185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:59.226729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.228355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.246402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 375/753] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:59.247997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.249890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.250703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.250988 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.251267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.260241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.261750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:59.279185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:59.280145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:59.281069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:59.285188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 379/754] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:59.286474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.366710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f16602-1eeb-4766-927c-6834b7c21c75 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.369058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:59.405982 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.407052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.407565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.408059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.409009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.410484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a215635-ef33-4a9a-be54-3e6ea986ab76 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.410875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.417659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 376/755] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:59.418837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.478144 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:59.491249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0e259-b34d-485a-a4f4-219f2fa23b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.492075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e259-b34d-485a-a4f4-219f2fa23b58 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.492362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e259-b34d-485a-a4f4-219f2fa23b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.492806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e259-b34d-485a-a4f4-219f2fa23b58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.518341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0e259-b34d-485a-a4f4-219f2fa23b58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.519846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0e259-b34d-485a-a4f4-219f2fa23b58 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-57c0e259-b34d-485a-a4f4-219f2fa23b58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:49:59.549584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:49:59.551417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:49:59.553280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:49:59.554111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 380/756] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:49:59.554227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505442f-9b28-4602-8092-f5e7bfc8ba5f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.558876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81245-2073-433a-ae93-356b473d5903 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.559332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81245-2073-433a-ae93-356b473d5903 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.559790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81245-2073-433a-ae93-356b473d5903 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.563605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 381/757] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:49:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:49:59.564617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81245-2073-433a-ae93-356b473d5903 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.623452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb3eee-c4f3-4862-bd12-b8201b438dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.624645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb3eee-c4f3-4862-bd12-b8201b438dff None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.625147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb3eee-c4f3-4862-bd12-b8201b438dff None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.625663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb3eee-c4f3-4862-bd12-b8201b438dff None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.627296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb3eee-c4f3-4862-bd12-b8201b438dff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:59.627866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb3eee-c4f3-4862-bd12-b8201b438dff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.630681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb3eee-c4f3-4862-bd12-b8201b438dff None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.654872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 377/758] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:59.657272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa39a69-368d-4f52-ba28-9cb5ab608069 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.657781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa39a69-368d-4f52-ba28-9cb5ab608069 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.658278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa39a69-368d-4f52-ba28-9cb5ab608069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.688014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8fa39a69-368d-4f52-ba28-9cb5ab608069 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:49:59.689656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8fa39a69-368d-4f52-ba28-9cb5ab608069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:49:59.723549 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81245-2073-433a-ae93-356b473d5903 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.775575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:59.778285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 378/759] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:49:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:49:59.779433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa39a69-368d-4f52-ba28-9cb5ab608069 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.787154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.789285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.790006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.790582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.792487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:59.793206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.796021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.812565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:49:59.821596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 382/760] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:59.823064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:49:59.897523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa39a69-368d-4f52-ba28-9cb5ab608069 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:49:59.938340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:49:59.939011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:49:59.939319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:49:59.939677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:49:59.940527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a411a835bce0475fae1db6e099dc2605) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:49:59.940948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:49:59.942428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:49:59.957170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 379/761] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/role_assignments?user.id=a411a835bce0475fae1db6e099dc2605&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:49:59.958767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:00.024044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ecdc0-23ee-45aa-87ba-118eedac6e78 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:00.130702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:00.133023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:00.133511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:00.133989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:00.182455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:00.195886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:00.197683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:00.199318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:00.201547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 383/762] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:00.203762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:00.263235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:00.264225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:00.302027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccea7e-c77e-4655-b94d-97a20112aa15 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:00.361254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:00.362408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:00.362907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:00.363404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:00.365070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=c813112306f840008f94dec2d2c844f4) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:00.366209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:00.369857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:00.389464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f389eb-af6b-40f0-9bec-c32228307049 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:00.396423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 380/763] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:49:59 2022] GET /identity/v3/role_assignments?user.id=c813112306f840008f94dec2d2c844f4&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 259 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:00.433332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:00.434044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:00.434334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:00.434618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:00.435423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:00.435736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:00.437214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:00.448229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 384/764] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:00.454310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:00.689010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf41db-bd0f-4318-ab7a-ad1e51d99069 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:00.769908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:00.771297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:00.772438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:00.772932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:00.794259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:00.797117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:00.832410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:00.833430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:00.834986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:00.834986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:00.834986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:00.834986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:00.836800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 381/765] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:00.842285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:01.097594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fd914-001e-4bf3-9b7b-9319e237e34c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:01.120331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:01.168343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:01.169580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:01.170168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:01.170685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:01.172097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:01.172912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:01.175629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:01.204786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 385/766] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:01.212375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:01.232288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:01.358429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785351-d61d-48ec-b181-7c861630a690 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:01.409475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd37a59-2556-4457-ab53-cb7562dee423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:01.410700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd37a59-2556-4457-ab53-cb7562dee423 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:01.411219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd37a59-2556-4457-ab53-cb7562dee423 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:01.411696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd37a59-2556-4457-ab53-cb7562dee423 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:01.425809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd37a59-2556-4457-ab53-cb7562dee423 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:01.427675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd37a59-2556-4457-ab53-cb7562dee423 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-6cd37a59-2556-4457-ab53-cb7562dee423 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:01.448499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:01.449762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:01.450707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:01.457106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 382/767] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:50:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:01.461998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:01.728374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fa51-46ab-4c72-87b2-40d35d281c23 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:01.728625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fa51-46ab-4c72-87b2-40d35d281c23 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:01.728876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fa51-46ab-4c72-87b2-40d35d281c23 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:01.730552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 383/768] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:01.736481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9a84-f68c-4da3-88f1-1d64a0ce85e9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:01.736812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9a84-f68c-4da3-88f1-1d64a0ce85e9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:01.737061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9a84-f68c-4da3-88f1-1d64a0ce85e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:01.753034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6e0a-492b-4ba7-b157-da33ce04e6d8 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:01.755590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e11c9a84-f68c-4da3-88f1-1d64a0ce85e9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:01.759976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e11c9a84-f68c-4da3-88f1-1d64a0ce85e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:01.837654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:01.838339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:01.838616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:01.838890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:01.839737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:01.840047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:01.841538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:01.866488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 386/769] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:50:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:01.872594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:01.902584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:01.916099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 384/770] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:01.941474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c9a84-f68c-4da3-88f1-1d64a0ce85e9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:01.949726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:02.098173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b47d7-29a6-4631-9678-7f8c804b6992 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.105394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c9a84-f68c-4da3-88f1-1d64a0ce85e9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.148082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035020c6-ad9b-4f9e-9158-db2f938a0344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.148805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035020c6-ad9b-4f9e-9158-db2f938a0344 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.149120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035020c6-ad9b-4f9e-9158-db2f938a0344 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.149401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035020c6-ad9b-4f9e-9158-db2f938a0344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.151818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.152899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.153518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.153994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] PATH_INFO: `/v3/services/network` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.155006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.156760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:02.170845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:02.171883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:50:02.173027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:02.173773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035020c6-ad9b-4f9e-9158-db2f938a0344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.173773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035020c6-ad9b-4f9e-9158-db2f938a0344 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.176333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 385/771] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:01 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:02.184291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-035020c6-ad9b-4f9e-9158-db2f938a0344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:02.197165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:02.198870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:02.200625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:02.200625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:02.200625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:02.200625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:02.203260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 387/772] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:02.205071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fca295-448c-48c9-a2be-6d9c30307be3 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.205368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fca295-448c-48c9-a2be-6d9c30307be3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.205649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fca295-448c-48c9-a2be-6d9c30307be3 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.212206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 388/773] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:02.212206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fca295-448c-48c9-a2be-6d9c30307be3 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.266717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:02.310329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a209-8fe4-4a4a-840d-1748c6110054 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.330317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:02.349081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3dbbd-39c0-4c1d-be34-3445106a5f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.349463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3dbbd-39c0-4c1d-be34-3445106a5f95 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.350556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3dbbd-39c0-4c1d-be34-3445106a5f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.350892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3dbbd-39c0-4c1d-be34-3445106a5f95 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.351717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3dbbd-39c0-4c1d-be34-3445106a5f95 None admin] RBAC: Adding query filter params (name=network) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:02.352050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3dbbd-39c0-4c1d-be34-3445106a5f95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.353491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3dbbd-39c0-4c1d-be34-3445106a5f95 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.363829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 386/774] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:02.363829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d604dc-d1e5-468b-ad6b-e0d8993e296c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.363829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d604dc-d1e5-468b-ad6b-e0d8993e296c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.363829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d604dc-d1e5-468b-ad6b-e0d8993e296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.364394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fca295-448c-48c9-a2be-6d9c30307be3 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.382539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-45d604dc-d1e5-468b-ad6b-e0d8993e296c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:02.384108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-45d604dc-d1e5-468b-ad6b-e0d8993e296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:02.408464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.409582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.410138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.410617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.411681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:02.412284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.414246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.431610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 389/775] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:02.432899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.479534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 387/776] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:02.479534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d604dc-d1e5-468b-ad6b-e0d8993e296c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.553936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:02.590823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d933688-6837-4a16-a3f9-64f57fa36d93 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.604659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d604dc-d1e5-468b-ad6b-e0d8993e296c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.632814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.633894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.634441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.635052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.635983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:02.636496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.638364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.643627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.644451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.644738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.645081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.651084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 390/777] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:02.652358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.653958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.655577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.657282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:02.672096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:02.673368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:02.674256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:02.679566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 388/778] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:02.680597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.783102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4f149-371d-49e4-980f-e15148b1d2a7 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.796722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b11f9-eec7-4c1b-83d8-2e9feab21d2f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:02.815335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:02.827344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.828604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.829148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.829810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.830920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.832627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.836189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:02.836865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:02.837244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:02.837526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:02.838357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] RBAC: Adding query filter params (interface=public, service_id=d561179906fd4ab0bff48e20654de5f3, region_id=RegionOne) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:02.838646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:02.840033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:02.848382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:02.849418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 07:50:02.850307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:02.851088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 389/779] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/endpoints?service_id=d561179906fd4ab0bff48e20654de5f3&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:02.851088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.857915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 391/780] 10.222.0.43 () {56 vars in 1129 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:02.857915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:02.949609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:02.986909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e25f8-c5df-4b1d-9533-0a254b4f8496 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:03.001578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db5578-c47e-47d4-a9dd-e27c6c731200 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:03.038459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:03.039313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:03.039775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.040006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.040262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.040534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.040899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.041245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:03.041661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.042630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:03.042916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:03.043422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:03.046040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:03.057426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 392/781] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:03.058437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:03.069487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 390/782] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:03.071349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:03.129850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:03.208584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51465b2-965b-41c3-a789-e9621114a97c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:03.253264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330d899-fabc-49bb-831a-34b8ffc25618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:03.255339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d899-fabc-49bb-831a-34b8ffc25618 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.256390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d899-fabc-49bb-831a-34b8ffc25618 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.256875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d899-fabc-49bb-831a-34b8ffc25618 None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/a267b6f64f5f49b8b3f5837e701bc216/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.259494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330d899-fabc-49bb-831a-34b8ffc25618 None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=a267b6f64f5f49b8b3f5837e701bc216, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:03.261638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330d899-fabc-49bb-831a-34b8ffc25618 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:03.285975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 391/783] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:50:03 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/a267b6f64f5f49b8b3f5837e701bc216/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:50:03.355255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6c913-54ec-437c-a846-819df26bd4b1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:03.373974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9886-dcaa-4026-80aa-0573f8503f38 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.374921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9886-dcaa-4026-80aa-0573f8503f38 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.375428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9886-dcaa-4026-80aa-0573f8503f38 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.379340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 392/784] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:03.386878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791822a2-b5d0-44ad-b953-b9cbd00f38aa None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.387487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791822a2-b5d0-44ad-b953-b9cbd00f38aa None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.387968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791822a2-b5d0-44ad-b953-b9cbd00f38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.404165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:03.410252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-791822a2-b5d0-44ad-b953-b9cbd00f38aa None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:03.413619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-791822a2-b5d0-44ad-b953-b9cbd00f38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:03.430088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:03.431246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.431735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.432211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.433633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:03.434200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:03.436903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:03.448590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 393/785] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jun 29 07:50:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:03.454827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:03.530651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 393/786] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:03.579751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791822a2-b5d0-44ad-b953-b9cbd00f38aa None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:03.594698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4749fb1-99e6-4262-aa38-d039e0b14366 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:03.645098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:03.645098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.645098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.645098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.645496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:03.645496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:03.648339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:03.661183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 394/787] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jun 29 07:50:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:03.667316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:03.669525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:03.875449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:03.884060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791822a2-b5d0-44ad-b953-b9cbd00f38aa None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:03.957102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17922237-fa38-46de-91f4-10067b576b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:03.958887 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17922237-fa38-46de-91f4-10067b576b85 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.960429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17922237-fa38-46de-91f4-10067b576b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.960928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17922237-fa38-46de-91f4-10067b576b85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.972881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e02a44-5d42-419d-8d34-3c22babf3050 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:03.982906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17922237-fa38-46de-91f4-10067b576b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-17922237-fa38-46de-91f4-10067b576b85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:03.991446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:03.992860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:03.993724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 394/788] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:03.997026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab7e7-b426-44a8-9896-2a9fc1ecfa87 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:03.997026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab7e7-b426-44a8-9896-2a9fc1ecfa87 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:03.997026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab7e7-b426-44a8-9896-2a9fc1ecfa87 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:03.997026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 395/789] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:04.002118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e231f3-cd60-4b27-851c-80bc338ab8ef None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:04.002668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e231f3-cd60-4b27-851c-80bc338ab8ef None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:04.003152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e231f3-cd60-4b27-851c-80bc338ab8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:04.028662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a0e231f3-cd60-4b27-851c-80bc338ab8ef None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:04.031519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a0e231f3-cd60-4b27-851c-80bc338ab8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:04.071348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:04.072876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:04.073420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:04.073915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:04.076162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:04.080621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:04.123556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 395/790] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:04.125430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:04.187671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 396/791] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:04.211973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e231f3-cd60-4b27-851c-80bc338ab8ef None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:04.245820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:04.270638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad8616-40f8-4d06-9dc6-81a548132ace None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:04.291723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:04.320736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:04.322581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:04.323293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:04.325008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:04.326005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:04.326706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:04.329107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:04.353601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 396/792] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:04.358432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:04.432841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e231f3-cd60-4b27-851c-80bc338ab8ef None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:04.484717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:04.486012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:04.486199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:04.486514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:04.500376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:04.509686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:04.511323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:50:04.512166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:04.512808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 397/793] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:04.517955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:04.563021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ba7be-c027-47de-8fca-961ef08a66cc None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:04.605204 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:04.606391 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:04.606869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:04.607354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:04.614401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:04.622828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:04.624756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:04.646396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792534b2-ee24-477f-9727-cb23607bc346 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:04.655574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:04.657295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:04.658986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:04.658986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:04.658986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:04.658986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:04.664196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 397/794] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:04.670817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:04.689938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:04.690739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:04.691026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:04.691356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:04.692205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:04.692524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:04.695295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:04.705713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 398/795] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:04.711899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:04.754227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:04.931083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cbb00-d0fe-49b8-be1b-2073e6d8cfa7 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:04.960135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57b6b-40b4-4c10-b03f-1ba1f94f7f86 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:05.004709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:05.007858 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:05.008559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:05.009274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:05.010793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:05.011574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:05.014026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:05.023539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:05.024376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:05.024766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:05.024921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:05.041283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 398/796] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:05.044083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:05.057012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:05.057279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:05.057279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:05.091772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:05.093252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:05.094309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:05.094309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:05.094309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:05.094309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:05.094552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 399/797] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:05.100229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:05.260160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:05.280603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb082937-ea22-442b-9717-fa895169f3e3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:05.333730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160eaaf-c4b4-4094-ab2c-28644aebf3af None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:05.354001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:05.354825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:05.355114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:05.355396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:05.364456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:05.366992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:05.383634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:05.385491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:50:05.387402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:05.389355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 399/798] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:05.391060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:05.405830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:05.406534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:05.406823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:05.407101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:05.407852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:05.408163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:05.409588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:05.432604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 400/799] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:05.439031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:05.562986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:05.574580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7be9a6-50cb-435e-9c40-d57ac8b775a1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:05.614953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:05.616261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:05.616706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:05.617159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:05.618234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:05.618695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:05.620455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:05.659101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 400/800] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jun 29 07:50:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:05.665843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:05.699974 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:05.708759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd67ba-db65-4e5d-b7df-b02ff937839b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:05.850381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:05.851925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:05.852593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:05.853347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:05.883219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:05.886440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:05.916227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:05.918456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:05.920613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:05.925005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 401/801] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:50:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 487 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:05.934957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:05.952729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abad64-6d2f-4fa3-97ab-ed477d92701e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:06.034073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bc9a7-a2ea-4f47-bde2-fbdd6bddcc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:06.035415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bc9a7-a2ea-4f47-bde2-fbdd6bddcc64 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.035997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bc9a7-a2ea-4f47-bde2-fbdd6bddcc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.036499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bc9a7-a2ea-4f47-bde2-fbdd6bddcc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.067548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bc9a7-a2ea-4f47-bde2-fbdd6bddcc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:06.075353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bc9a7-a2ea-4f47-bde2-fbdd6bddcc64 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-434bc9a7-a2ea-4f47-bde2-fbdd6bddcc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:06.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:06.114011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:06.114912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:06.114912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:06.114912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:06.114912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:06.115657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 401/802] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:06.118062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd978b09-85cf-44f6-b6d5-d09ccb92e4ba None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.121017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd978b09-85cf-44f6-b6d5-d09ccb92e4ba None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.121017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd978b09-85cf-44f6-b6d5-d09ccb92e4ba None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.122019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 402/803] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:06.124203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7466a9b-434e-43b8-8883-86890dfcf396 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.124705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7466a9b-434e-43b8-8883-86890dfcf396 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.125244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7466a9b-434e-43b8-8883-86890dfcf396 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.127746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 403/804] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:06.131985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7466a9b-434e-43b8-8883-86890dfcf396 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:06.143811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de3e3b-4a7f-4621-9c60-b165cb0b0807 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:06.164035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:06.193127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffaf1b-0a16-46a1-b424-741913622d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:06.193965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffaf1b-0a16-46a1-b424-741913622d00 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.194292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffaf1b-0a16-46a1-b424-741913622d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.194641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffaf1b-0a16-46a1-b424-741913622d00 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.195548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffaf1b-0a16-46a1-b424-741913622d00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:06.195862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffaf1b-0a16-46a1-b424-741913622d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:06.197668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffaf1b-0a16-46a1-b424-741913622d00 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:06.228941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 402/805] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jun 29 07:50:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:06.231260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a8f33-6526-4e36-8dd6-9e2f984d90c1 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.231759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a8f33-6526-4e36-8dd6-9e2f984d90c1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.232244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a8f33-6526-4e36-8dd6-9e2f984d90c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.254732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e17a8f33-6526-4e36-8dd6-9e2f984d90c1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:06.255211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:06.257178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e17a8f33-6526-4e36-8dd6-9e2f984d90c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:06.338741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7466a9b-434e-43b8-8883-86890dfcf396 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:06.371828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 403/806] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:06.374075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898925b-0ef4-4b3e-a558-578ea5f62cfe None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.374536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898925b-0ef4-4b3e-a558-578ea5f62cfe None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.375043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898925b-0ef4-4b3e-a558-578ea5f62cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.400794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:06.402491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.403364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.404150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.405009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e898925b-0ef4-4b3e-a558-578ea5f62cfe None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:06.406151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:06.406948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e898925b-0ef4-4b3e-a558-578ea5f62cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:06.407265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:06.410176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:06.431895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 404/807] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:06.431895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:06.532245 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 404/808] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:06.533428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898925b-0ef4-4b3e-a558-578ea5f62cfe None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:06.585108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee40ef7-656e-4a53-b29d-d89530bc76d8 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:06.595149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:06.623198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:06.649216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:06.650558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.651027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.651485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.670210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:06.673003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:06.696884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898925b-0ef4-4b3e-a558-578ea5f62cfe None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:06.701557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:06.703322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:06.706521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:06.706521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:06.706521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:06.706521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:06.711418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 405/809] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:06.711956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:06.741931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:06.742929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.743334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.743732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] PATH_INFO: `/v3/services/network` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.744531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:06.746199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:06.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:06.765417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:06.767073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 405/810] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:06 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:06.770119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:06.844048 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8f90a-0a81-459a-b96a-a70dd1fba45b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:06.885352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:06.886093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.886403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.886669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.895200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:06.895455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:06.897955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:06.906945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8484bc-fd17-4bd3-8836-9ffafdcf616e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:06.919040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:06.921238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:06.923898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:06.925478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 406/811] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:06.926494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:06.962945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:06.963295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:06.964705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:06.965227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:06.965722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:06.985625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:07.001196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:07.002217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:07.003061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:07.003705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 406/812] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:07.004920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.060569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f535d-e83f-4431-95c1-4dc0ec54fb8e None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.103825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.104500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.104798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.105090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.105811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:07.106227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.107638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.128140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 407/813] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:07.128140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.145575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ae0e-ccf2-4080-a4a7-fe9dad39bdd3 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.191932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.192619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.192909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.193750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.194502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] RBAC: Adding query filter params (name=network) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:07.194810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.196217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.202773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 407/814] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:07.203627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.226147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:07.227194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:07.235763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc4df7-c866-49e3-bf20-b5e31ae8b815 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.275362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.276133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.276537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.276885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.277796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:07.278203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.280137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.294585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 408/815] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:07.295454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.323448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9c22d-1524-45e3-8714-5a3188e89024 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.361570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafb2ed-f417-4acc-8bd7-15f992574150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.362260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb2ed-f417-4acc-8bd7-15f992574150 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.362549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb2ed-f417-4acc-8bd7-15f992574150 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.362829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb2ed-f417-4acc-8bd7-15f992574150 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.363599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb2ed-f417-4acc-8bd7-15f992574150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:07.363914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb2ed-f417-4acc-8bd7-15f992574150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.365851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb2ed-f417-4acc-8bd7-15f992574150 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.375042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 408/816] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:07.376406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02900e38-2718-4a92-b7f8-8eac1fd5b54e None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.376701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02900e38-2718-4a92-b7f8-8eac1fd5b54e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.377005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02900e38-2718-4a92-b7f8-8eac1fd5b54e None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.378490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 409/817] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:07.379062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02900e38-2718-4a92-b7f8-8eac1fd5b54e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.415495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c6763-495b-4725-be07-fc2554448fb0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.457971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.458783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.459762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.460071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.468964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.470468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.471686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:07.485663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:07.487284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 409/818] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:07.488963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.503924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02900e38-2718-4a92-b7f8-8eac1fd5b54e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.511036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:07.548081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.549178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.549609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.550037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.551271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:07.551786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.556194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.562422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 410/819] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:07.563415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.629394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f66f56-0bee-43c2-be6b-0130ee598aae None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.669931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a53822-e33e-4ee0-af85-44721cbd1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.670781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a53822-e33e-4ee0-af85-44721cbd1170 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.671065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a53822-e33e-4ee0-af85-44721cbd1170 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.671341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a53822-e33e-4ee0-af85-44721cbd1170 None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/a411a835bce0475fae1db6e099dc2605/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.673715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a53822-e33e-4ee0-af85-44721cbd1170 None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=a411a835bce0475fae1db6e099dc2605, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.675176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a53822-e33e-4ee0-af85-44721cbd1170 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.690777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 410/820] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:50:07 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/a411a835bce0475fae1db6e099dc2605/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:50:07.691468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a064e84-04f4-4edd-b37a-932103a88cc4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:07.693090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959074-f08f-46b5-b453-45ae1f219a58 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.693684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959074-f08f-46b5-b453-45ae1f219a58 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.694167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959074-f08f-46b5-b453-45ae1f219a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.722172 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-75959074-f08f-46b5-b453-45ae1f219a58 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:07.723544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-75959074-f08f-46b5-b453-45ae1f219a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:07.739214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:07.740552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:07.741029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:07.741483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:07.758652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:07.761719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:07.765159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:07.786071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:07.788576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:07.790876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:07.790876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:07.790876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:07.790876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:07.790876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 411/821] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:07.791494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.842384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 411/822] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:07.844183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959074-f08f-46b5-b453-45ae1f219a58 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:07.848837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:07.989089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7683be-f73e-4459-a1fa-6254332a7418 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.031933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959074-f08f-46b5-b453-45ae1f219a58 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.037567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.038697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.039208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.039530 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.041027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:08.041499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.044192 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.073382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 412/823] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:08.074903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:08.084207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.085246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.085670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.086024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.087395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.089914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.133017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 412/824] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:50:08.133017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:08.171400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:08.258654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:08.272838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf703d5-9f84-4067-8ac0-0f14100b9c25 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.335876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.337361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12220bc-f0c6-41c5-a0b0-6c6206e6dee0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.337639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.338291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.338917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.340346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:08.341076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.343673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.376575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 413/825] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:08.376575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:08.397869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.398704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.398983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.399249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.399916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.404194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:08.416282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:08.418252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:08.419889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 413/826] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:08.423058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:08.555155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784acf3-f333-4ccd-a251-2d6f2322df57 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.558952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217bdf-4603-4622-ac0f-affec8fbd634 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.577284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:08.587965 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:08.606170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.607304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.607798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.608352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.611989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.612911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.613223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.613499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/c813112306f840008f94dec2d2c844f4/roles/f36cc14a79404ebbaa216562f0f34fa1` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.620404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.622151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.632002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=c813112306f840008f94dec2d2c844f4, role_id=f36cc14a79404ebbaa216562f0f34fa1)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.633524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:08.642315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:08.644745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 414/827] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:50:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:08.647116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:08.665447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 414/828] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:50:08 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/c813112306f840008f94dec2d2c844f4/roles/f36cc14a79404ebbaa216562f0f34fa1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:50:08.665447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:08.841064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ed193-0c84-404e-b921-b39bf872369e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.870379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603dcd7-7afc-49b9-8608-f4e661b8c09b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:08.887475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.888462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.888904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.889350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.890483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:08.890954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.892735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.915141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 415/829] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:50:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:08.921133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:08.931169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:08.932864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:08.933384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:08.933848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:08.935348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:08.936300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:08.938422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:08.944596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:08.947798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 415/830] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:08.956701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:09.017753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:09.100729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c87ea-d059-4dea-980d-489c57253aad None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:09.113781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c193-7902-420b-a5ef-9ec98e08112a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:09.150125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:09.150835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:09.151134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:09.151483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:09.184938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:09.186432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:09.187076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:09.187833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:09.189391 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:09.190087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:09.192926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:09.202258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 416/831] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:09.267807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:09.271889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:09.306397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:09.492650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 416/832] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 07:50:09.498767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:09.700656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65db0a-8796-4b73-bcfe-2916c850844a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:09.723770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:09.772666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:09.774274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:09.774775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:09.775267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:09.798634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:09.801373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:09.838260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:09.839217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:09.840121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:09.840121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:09.840121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:09.840121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:09.841817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 417/833] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:09.846914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:09.973701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e436a-b96f-4976-af19-53060c98d523 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:09.995551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:10.018989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:10.020136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.020635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.028183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.028183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:10.028183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:10.028183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:10.048694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 417/834] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:10.054542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:10.201623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d6201-fab2-45c4-aeb9-50174160acba None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:10.269893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2221b8-94cc-479f-9cdd-9e72955b4b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:10.271379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2221b8-94cc-479f-9cdd-9e72955b4b2c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.271886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2221b8-94cc-479f-9cdd-9e72955b4b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.272593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2221b8-94cc-479f-9cdd-9e72955b4b2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.286521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2221b8-94cc-479f-9cdd-9e72955b4b2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:10.290524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2221b8-94cc-479f-9cdd-9e72955b4b2c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-5d2221b8-94cc-479f-9cdd-9e72955b4b2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:10.320928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:10.321949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:10.322873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:10.323824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 418/835] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:10.329966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:10.370470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd49c-6ae0-429d-bb4c-db8c623919cb None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.370470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd49c-6ae0-429d-bb4c-db8c623919cb None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.370470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd49c-6ae0-429d-bb4c-db8c623919cb None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.370851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 419/836] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:10.383343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3094c-5ad5-4908-b29f-7837bc4fc599 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.384043 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3094c-5ad5-4908-b29f-7837bc4fc599 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.384613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3094c-5ad5-4908-b29f-7837bc4fc599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.409731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9af3094c-5ad5-4908-b29f-7837bc4fc599 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:10.410972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9af3094c-5ad5-4908-b29f-7837bc4fc599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:10.433093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:10.501018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 420/837] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:10.508205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91745e74-fe51-4b68-b90d-864fd8ce12c2 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:10.537762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3094c-5ad5-4908-b29f-7837bc4fc599 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:10.555586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:10.557252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.558207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.559180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.561073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:10.562089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:10.565475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:10.592748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 418/838] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:10.600558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:10.658927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3094c-5ad5-4908-b29f-7837bc4fc599 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:10.673451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:10.702020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:10.702857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.703201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.703908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.717925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:10.721006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc13b1-77c9-4edc-bd79-d7273ab9f797 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:10.726813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:10.727810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:10.729114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 421/839] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:10.734144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:10.752310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:10.774855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:10.776875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.777501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.777991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.779412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a267b6f64f5f49b8b3f5837e701bc216) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:10.779990 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:10.782670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:10.795401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 419/840] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:10 2022] GET /identity/v3/role_assignments?user.id=a267b6f64f5f49b8b3f5837e701bc216&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:10.874895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9c926-1133-4955-b2e9-ebaa00ccea4b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:10.922022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:10.923387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:10.923918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:10.924399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:10.925506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:10.926040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:10.928667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:10.939675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 422/841] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:10.947946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:11.069257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a073927-e4d6-413c-aae1-ac97bb210cb9 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:11.077188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:11.108202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:11.109045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:11.109395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:11.109684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:11.121600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:11.123155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:11.143439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:11.144393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:11.145755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:11.145755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:11.145755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:11.145755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:11.147965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 420/842] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:11.154556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:11.282217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:11.317364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c35208-1e87-4783-aebd-5315ef3e4924 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:11.356169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd791681-8692-430c-927b-1f070a96641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:11.357239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd791681-8692-430c-927b-1f070a96641f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:11.357726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd791681-8692-430c-927b-1f070a96641f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:11.358199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd791681-8692-430c-927b-1f070a96641f None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:11.359153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd791681-8692-430c-927b-1f070a96641f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:11.359658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd791681-8692-430c-927b-1f070a96641f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:11.361297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd791681-8692-430c-927b-1f070a96641f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:11.376346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 423/843] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:11.386295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:11.584866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fe33c-0bff-46bb-93a9-5f854dcdd6a0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:11.655241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4888b3d-4dd6-417c-9c6d-aab04105561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:11.656628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4888b3d-4dd6-417c-9c6d-aab04105561b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:11.657268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4888b3d-4dd6-417c-9c6d-aab04105561b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:11.657785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4888b3d-4dd6-417c-9c6d-aab04105561b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:11.674204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4888b3d-4dd6-417c-9c6d-aab04105561b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:11.677027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4888b3d-4dd6-417c-9c6d-aab04105561b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-f4888b3d-4dd6-417c-9c6d-aab04105561b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:11.707695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:11.708752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 07:50:11.709922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:11.711053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 421/844] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:11.719660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd791681-8692-430c-927b-1f070a96641f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:11.736554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de169c-111d-48a6-a2d6-b66859c4a776 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:11.737238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de169c-111d-48a6-a2d6-b66859c4a776 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:11.738422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de169c-111d-48a6-a2d6-b66859c4a776 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:11.742127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 422/845] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:11.747798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75eda-5ead-4e81-8470-0ab4c3ce7f2f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:11.748387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75eda-5ead-4e81-8470-0ab4c3ce7f2f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:11.748906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75eda-5ead-4e81-8470-0ab4c3ce7f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:11.772736 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-97e75eda-5ead-4e81-8470-0ab4c3ce7f2f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:11.774962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-97e75eda-5ead-4e81-8470-0ab4c3ce7f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:11.789919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:11.889046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 423/846] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:11.906541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e75eda-5ead-4e81-8470-0ab4c3ce7f2f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:11.950746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd791681-8692-430c-927b-1f070a96641f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:11.991395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:11.992463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:11.992968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:11.993477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:11.994535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:11.995050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:11.996713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:11.999276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:12.018376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 424/847] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jun 29 07:50:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:12.027841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:12.075211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e75eda-5ead-4e81-8470-0ab4c3ce7f2f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:12.117793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:12.118766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.119052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.119330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.132260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:12.141138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:12.145770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:50:12.148698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:12.151903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 424/848] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:50:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:12.151903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:12.155685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:12.247146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6b8e-e86c-4319-8624-1d11ad4aded9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:12.281510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6a92b-a30b-48ef-bf5d-e32f280d0fd6 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:12.314451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:12.315707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.316222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.316728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.331768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c47f8-6ed4-4901-ba5e-a1d61927f266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:12.332635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c47f8-6ed4-4901-ba5e-a1d61927f266 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.333097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c47f8-6ed4-4901-ba5e-a1d61927f266 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.333525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c47f8-6ed4-4901-ba5e-a1d61927f266 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.334538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c47f8-6ed4-4901-ba5e-a1d61927f266 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:12.335070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c47f8-6ed4-4901-ba5e-a1d61927f266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:12.336821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c47f8-6ed4-4901-ba5e-a1d61927f266 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:12.347903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 425/849] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:50:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:12.350889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb68ea2-abca-46d1-9b79-1fa75cf98d8c None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.351402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb68ea2-abca-46d1-9b79-1fa75cf98d8c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.351894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb68ea2-abca-46d1-9b79-1fa75cf98d8c None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.354847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 426/850] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:12.361025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804ee9-4457-406f-94dd-b8e08596794c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.361500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804ee9-4457-406f-94dd-b8e08596794c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.361969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804ee9-4457-406f-94dd-b8e08596794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.384578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-40804ee9-4457-406f-94dd-b8e08596794c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:12.387147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-40804ee9-4457-406f-94dd-b8e08596794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:12.443143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:12.459719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:12.462205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:12.492335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:12.505266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 427/851] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:12.508848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40804ee9-4457-406f-94dd-b8e08596794c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:12.655837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40804ee9-4457-406f-94dd-b8e08596794c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:12.665814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 425/852] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 07:50:12.668095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:12.700672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:12.701523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.701801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.702080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.716191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:12.717714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:12.737676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:12.739571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:12.744161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:12.744161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:12.744161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:12.744161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:12.744161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 428/853] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:12.744161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:12.806121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:12.812507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:12.824263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc9d58-3109-46b0-9cd7-20c7d027dc06 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:12.863440 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:12.864131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.864427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.864709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.866358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:12.868149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:12.889935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 426/854] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:50:12 2022] POST /identity/v3/services => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:50:12.889935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:12.910652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4393204-18db-451e-9ac1-b468c37ef4e4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:12.982408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:12.983689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:12.984003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:12.984273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:12.999177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.001321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:13.025940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:13.027758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:13.030435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:13.030435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:13.030435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:13.030435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:13.032179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 429/855] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:13.033234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:13.174983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:13.186983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07642138-d6ba-498d-b5cc-c96ce0ea7c45 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:13.237806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1048d6d-bbb3-480e-9e25-5f29189849aa None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:13.263824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:13.265251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.265907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.266525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.268010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:13.268704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.272129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:13.279903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:13.280578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.280859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.281160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.281893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:13.282194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.283568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:13.297269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 427/856] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:13.298084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 430/857] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:13.303642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:13.303816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:13.322311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:13.447811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a7ff5-40bd-48ca-833c-f4f0160ea8c3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:13.472579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3bd12-3783-43ab-98a5-e9fa4b597d4c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:13.510957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f52ab8-db4d-41b8-b8ab-7a49a3a08833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:13.512240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f52ab8-db4d-41b8-b8ab-7a49a3a08833 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.512746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f52ab8-db4d-41b8-b8ab-7a49a3a08833 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.513293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f52ab8-db4d-41b8-b8ab-7a49a3a08833 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.523295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f52ab8-db4d-41b8-b8ab-7a49a3a08833 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.524902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f52ab8-db4d-41b8-b8ab-7a49a3a08833 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:13.528109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:13.529346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.529853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.530358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c0f52ab8-db4d-41b8-b8ab-7a49a3a08833 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:13.540659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:13.550358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 428/858] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a64a-479e-4f67-8d26-58116556947b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.552095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a64a-479e-4f67-8d26-58116556947b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.554670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a64a-479e-4f67-8d26-58116556947b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.554670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 429/859] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:13.555055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.556328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:13.557695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b8da-d673-4f4a-8a45-2f50b73fb7c0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.558786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b8da-d673-4f4a-8a45-2f50b73fb7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.559578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b8da-d673-4f4a-8a45-2f50b73fb7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:13.572946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:13.573997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:13.574975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:13.581326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 431/860] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:50:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:13.581326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:13.581675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4e73b8da-d673-4f4a-8a45-2f50b73fb7c0 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:13.581675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4e73b8da-d673-4f4a-8a45-2f50b73fb7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:13.604042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:13.697063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 430/861] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:13.697063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b8da-d673-4f4a-8a45-2f50b73fb7c0 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:13.698299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:13.720886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826ca8f-dfe6-40a3-91ff-e250a064eafc None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:13.763503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:13.764569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.764921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.765275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.766341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:13.766764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.768238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:13.783957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 432/862] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:13.784926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:13.859080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b8da-d673-4f4a-8a45-2f50b73fb7c0 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:13.889228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:13.897432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:13.898111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.898391 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.898664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.899509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:13.899825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.901315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:13.921032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 431/863] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jun 29 07:50:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:13.921596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:13.932208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c8be2-44b4-4832-a4e9-52e0c7e34aa6 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:13.988656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:13.989943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:13.990873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:13.991532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:13.993009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:13.994738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:14.003706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:14.015279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:14.017036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:14.018753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 433/864] 10.222.0.43 () {56 vars in 1132 bytes} [Wed Jun 29 07:50:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:14.020426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:14.067278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fe05e-3c4a-406b-99d2-ebb100bb0633 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:14.109936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:14.110608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:14.110889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:14.111168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:14.112002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a411a835bce0475fae1db6e099dc2605) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:14.112313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:14.113886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:14.129023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 432/865] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:13 2022] GET /identity/v3/role_assignments?user.id=a411a835bce0475fae1db6e099dc2605&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:14.129023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:14.167321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f303737-caf1-4f8f-880f-698b95a72f6d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:14.217351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:14.219131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:14.219645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:14.220148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:14.245723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:14.314342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764a68a-ac9b-4798-ac06-af37cf6c40da None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:14.333151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:14.335913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:14.352764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:14.373926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:14.375017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:14.375450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:14.375733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:14.376619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:14.377029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:14.378813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:14.385450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 433/866] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:14.392672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:14.475258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 434/867] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:14 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:50:14.488033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:14.545336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:14.558607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664ec48-710b-4762-ae5f-c6d8018693d1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:14.618314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:14.619519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:14.619812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:14.620299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:14.621122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:14.621684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:14.625577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:14.638845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2ac9d-a1e5-4ead-8e43-5781239e4dfb None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:14.641075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 434/868] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:14.689157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:14.690065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:14.690342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:14.690628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:14.704687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:14.706251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:14.729433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:14.730654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:14.732140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:14.732140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:14.732140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:14.732140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:14.732140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 435/869] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:14.740753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:14.819567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:14.879703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aa1af-2783-415f-b997-041c2b82fc6a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:14.921241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:14.921928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:14.922207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:14.922487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:14.923485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:14.923794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:14.925225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:14.939395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 435/870] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:14.946138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:15.077487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:15.087416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0da75-f6ce-441c-8513-3659026dfd88 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:15.127428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb7399-2175-4cc0-aef2-a37605a0fe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:15.128250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb7399-2175-4cc0-aef2-a37605a0fe14 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:15.128588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb7399-2175-4cc0-aef2-a37605a0fe14 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:15.128867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb7399-2175-4cc0-aef2-a37605a0fe14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:15.137121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb7399-2175-4cc0-aef2-a37605a0fe14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:15.138600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb7399-2175-4cc0-aef2-a37605a0fe14 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-71bb7399-2175-4cc0-aef2-a37605a0fe14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:15.155148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:15.156129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:15.157058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 436/871] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:15.163379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:15.260969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a27e9-07be-4f89-a8b0-fd79c727ba97 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:15.261923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a27e9-07be-4f89-a8b0-fd79c727ba97 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:15.262690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a27e9-07be-4f89-a8b0-fd79c727ba97 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:15.265148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 437/872] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:15.270810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447e3e-1a6d-4952-8105-89a1135f23c9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:15.271506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447e3e-1a6d-4952-8105-89a1135f23c9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:15.272331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447e3e-1a6d-4952-8105-89a1135f23c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:15.287567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8c052-e744-4a90-b31c-f0b4a692ccb2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:15.293450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c4447e3e-1a6d-4952-8105-89a1135f23c9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:15.295759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c4447e3e-1a6d-4952-8105-89a1135f23c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:15.304153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:15.333445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:15.335086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:15.335459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:15.335842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:15.336663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:15.337066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:15.338726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:15.355479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 436/873] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:15.361256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:15.406827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 438/874] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:15.452759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4447e3e-1a6d-4952-8105-89a1135f23c9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:15.488539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e745de-3a21-43bb-803e-17079d4d3cf2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:15.559174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:15.560327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:15.560871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:15.561437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:15.562579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=c813112306f840008f94dec2d2c844f4) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:15.563149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:15.564935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:15.573824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 437/875] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:15 2022] GET /identity/v3/role_assignments?user.id=c813112306f840008f94dec2d2c844f4&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:15.587574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:15.604907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4447e3e-1a6d-4952-8105-89a1135f23c9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:15.652066 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:15.653003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:15.653313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:15.653590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:15.654309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:15.655774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:15.669955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:15.671024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:15.672333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 439/876] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:15.679693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:15.726284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:15.809435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef662b8-990f-4c82-b706-d4c3382551cf None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:15.851628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:15.851819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:15.852046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:15.852261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:15.853120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:15.853428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:15.854905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:15.860728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 438/877] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:15.867338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:15.992714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d27ec-31fc-4ab9-bd4b-6c44dbf19963 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:16.033996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:16.034703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:16.034994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:16.035275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:16.036019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:16.036332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:16.038484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:16.045022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 440/878] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:16.054761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:16.223825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:16.228165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d9b18-7e1c-41d2-9cd5-ebfa06732508 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:16.266565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03238c-6074-4e2f-a6d2-1b1b5e57067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:16.267485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03238c-6074-4e2f-a6d2-1b1b5e57067d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:16.267911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03238c-6074-4e2f-a6d2-1b1b5e57067d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:16.268333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03238c-6074-4e2f-a6d2-1b1b5e57067d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:16.269353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03238c-6074-4e2f-a6d2-1b1b5e57067d None admin] RBAC: Adding query filter params (interface=public, service_id=13eabe3a10594c2fa1898e0a87a596c2, region_id=RegionOne) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:16.269812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03238c-6074-4e2f-a6d2-1b1b5e57067d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:16.271412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03238c-6074-4e2f-a6d2-1b1b5e57067d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:16.277970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 439/879] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:50:16 2022] GET /identity/v3/endpoints?service_id=13eabe3a10594c2fa1898e0a87a596c2&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:16.289004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:16.303470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:16.401383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75525ca7-a118-46d8-857c-7a0c05ccf4cb None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:16.440141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba109a26-2445-422a-9add-81f0f05ef52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:16.440875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba109a26-2445-422a-9add-81f0f05ef52d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:16.441192 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba109a26-2445-422a-9add-81f0f05ef52d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:16.441528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba109a26-2445-422a-9add-81f0f05ef52d None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:16.442222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba109a26-2445-422a-9add-81f0f05ef52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:16.444095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba109a26-2445-422a-9add-81f0f05ef52d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:16.450731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 441/880] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:50:16 2022] GET /identity/v3/services => generated 1011 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:50:16.647338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033466a3-58d5-4674-a1f8-06f01be98635 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:16.648381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033466a3-58d5-4674-a1f8-06f01be98635 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:16.648871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033466a3-58d5-4674-a1f8-06f01be98635 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:16.651347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 440/881] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:16.657508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a398f4-862b-424e-bfa3-bfbadf91c96e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:16.657999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a398f4-862b-424e-bfa3-bfbadf91c96e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:16.658288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a398f4-862b-424e-bfa3-bfbadf91c96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:16.679360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b5a398f4-862b-424e-bfa3-bfbadf91c96e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:16.680603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b5a398f4-862b-424e-bfa3-bfbadf91c96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:16.761776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 442/882] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:16.785249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033466a3-58d5-4674-a1f8-06f01be98635 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:16.900633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033466a3-58d5-4674-a1f8-06f01be98635 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:16.939297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:16.940517 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:16.941042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:16.941520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:16.942408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:16.944572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:16.957984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:16.959236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:16.960064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:16.963130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 441/883] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:16.969373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a398f4-862b-424e-bfa3-bfbadf91c96e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:16.973916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:17.091096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a398f4-862b-424e-bfa3-bfbadf91c96e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:17.128442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33c3c8-af0c-4367-acf6-0b2dff420c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:17.129177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33c3c8-af0c-4367-acf6-0b2dff420c0d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:17.129464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33c3c8-af0c-4367-acf6-0b2dff420c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:17.129742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33c3c8-af0c-4367-acf6-0b2dff420c0d None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:17.130533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33c3c8-af0c-4367-acf6-0b2dff420c0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:17.130849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33c3c8-af0c-4367-acf6-0b2dff420c0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:17.132447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33c3c8-af0c-4367-acf6-0b2dff420c0d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:17.138195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 443/884] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:17.143876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:17.185574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:17.268864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358bdc2-fb0a-47dd-845b-6fb33a60cc3c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:17.306611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc73c9-7570-4aff-b652-5ebcaf2a307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:17.307296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc73c9-7570-4aff-b652-5ebcaf2a307c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:17.307578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc73c9-7570-4aff-b652-5ebcaf2a307c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:17.307855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc73c9-7570-4aff-b652-5ebcaf2a307c None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:17.308582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc73c9-7570-4aff-b652-5ebcaf2a307c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:17.308888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc73c9-7570-4aff-b652-5ebcaf2a307c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:17.310330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc73c9-7570-4aff-b652-5ebcaf2a307c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:17.316026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 442/885] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:17.483241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c860e06-79ed-4c50-b4a8-6a45e6ce637b None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:17.483658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c860e06-79ed-4c50-b4a8-6a45e6ce637b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:17.484151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c860e06-79ed-4c50-b4a8-6a45e6ce637b None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:17.487084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 444/886] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:17.495369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f30c8-b1f4-4cf3-8b79-b01f455b7244 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:17.495660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f30c8-b1f4-4cf3-8b79-b01f455b7244 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:17.495947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f30c8-b1f4-4cf3-8b79-b01f455b7244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:17.514907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-761f30c8-b1f4-4cf3-8b79-b01f455b7244 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:17.516111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-761f30c8-b1f4-4cf3-8b79-b01f455b7244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:17.591966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 443/887] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:17.633981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c860e06-79ed-4c50-b4a8-6a45e6ce637b None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:17.720011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:17.761871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c860e06-79ed-4c50-b4a8-6a45e6ce637b None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:17.804204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753561e5-f271-4bb8-adc0-bd7404dad4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:17.804902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753561e5-f271-4bb8-adc0-bd7404dad4b6 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:17.805220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753561e5-f271-4bb8-adc0-bd7404dad4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:17.805498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753561e5-f271-4bb8-adc0-bd7404dad4b6 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:17.806242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753561e5-f271-4bb8-adc0-bd7404dad4b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:17.807631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753561e5-f271-4bb8-adc0-bd7404dad4b6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:17.825188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 445/888] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:50:17 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:50:18.046432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14573e03-f933-4f53-8ae7-4cd1801ed28c None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:18.047414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14573e03-f933-4f53-8ae7-4cd1801ed28c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:18.048176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14573e03-f933-4f53-8ae7-4cd1801ed28c None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:18.050188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 444/889] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:18.057318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565538f-d7f1-4a23-b10d-a9c2604b50e9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:18.057942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565538f-d7f1-4a23-b10d-a9c2604b50e9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:18.058433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565538f-d7f1-4a23-b10d-a9c2604b50e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:18.079638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a565538f-d7f1-4a23-b10d-a9c2604b50e9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:18.081919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a565538f-d7f1-4a23-b10d-a9c2604b50e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:18.229520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 446/890] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:18.253474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14573e03-f933-4f53-8ae7-4cd1801ed28c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:18.282146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:18.372443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14573e03-f933-4f53-8ae7-4cd1801ed28c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:18.414176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:18.415428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:18.415948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:18.416443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:18.429507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:18.438422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:18.439430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:18.440278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:18.443876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 445/891] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jun 29 07:50:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:18.450367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565538f-d7f1-4a23-b10d-a9c2604b50e9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:18.527271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:18.583358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565538f-d7f1-4a23-b10d-a9c2604b50e9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:18.626826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280eaf9-a2bb-4ec6-b5de-ce3bc280e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:18.627433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280eaf9-a2bb-4ec6-b5de-ce3bc280e459 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:18.627715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280eaf9-a2bb-4ec6-b5de-ce3bc280e459 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:18.627994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280eaf9-a2bb-4ec6-b5de-ce3bc280e459 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:18.628767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280eaf9-a2bb-4ec6-b5de-ce3bc280e459 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:18.629114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280eaf9-a2bb-4ec6-b5de-ce3bc280e459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:18.630786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280eaf9-a2bb-4ec6-b5de-ce3bc280e459 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:18.640880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 447/892] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jun 29 07:50:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:18.647730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:18.728261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a12aa-819d-4c6a-88aa-0d5ab3e6b3f3 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:18.729002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a12aa-819d-4c6a-88aa-0d5ab3e6b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:18.729002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a12aa-819d-4c6a-88aa-0d5ab3e6b3f3 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:18.730827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 448/893] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:18.736315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386d76f-0010-4416-ae01-8d72f4fbcec8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:18.736586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386d76f-0010-4416-ae01-8d72f4fbcec8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:18.736954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386d76f-0010-4416-ae01-8d72f4fbcec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:18.756221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1386d76f-0010-4416-ae01-8d72f4fbcec8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:18.757585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1386d76f-0010-4416-ae01-8d72f4fbcec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:18.777761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:18.808060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf85d-64ca-44b4-9002-6ee89136cfde None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:18.857105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 449/894] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:18.863158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:18.864508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:18.865578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:18.866063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:18.877855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386d76f-0010-4416-ae01-8d72f4fbcec8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:18.884133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:18.885679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:18.905930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:18.909192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:18.912247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:18.912247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:18.912247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:18.912247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:18.912247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 446/895] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:18.923214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:18.971528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:19.112075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386d76f-0010-4416-ae01-8d72f4fbcec8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.130332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09bd4a-1fb8-41a9-b870-ae7450a1dca8 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.183198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:19.184429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:19.184907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:19.185423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:19.186829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:19.189467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:19.199591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:19.201115 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:19.201915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:19.202632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:19.204108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:19.204953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:19.207712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:19.210825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:19.214110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:19.216627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:19.217955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 450/896] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:19.226662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:19.245343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 447/897] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:19.248369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:19.381221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b9d28-b148-4584-949b-f7e378f20435 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.394550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:19.400078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af387-b271-48f3-b99d-0bb2592cb0da None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.427711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:19.428450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:19.428735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:19.429045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:19.429793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:19.430105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:19.431699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:19.437932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 451/898] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:19.441570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:19.442636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:19.443376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:19.443835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:19.448361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:19.454032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:19.455887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:19.473542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:19.474776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:19.475853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:19.483585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 448/899] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:50:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:19.484899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:19.554497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:19.603709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856f63b-c200-4aaa-b0c2-35ed44c1388c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.659445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:19.660907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:19.661742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:19.662648 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:19.664662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:19.665456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:19.668483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:19.675024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39ae69-9e1b-4fda-ae0d-836c004fcf63 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.704952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 449/900] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jun 29 07:50:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:19.713099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:19.748257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:19.749491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:19.750056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:19.750601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:19.751895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:19.752481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:19.756361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:19.773558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 452/901] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jun 29 07:50:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:19.774937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:19.850162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:19.901315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5017b3-4336-43ec-968c-e08825bbbbb3 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.925198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:19.971994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b7e7-5cab-45f6-8ffe-5d0e08278abf None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:19.973266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:19.974736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:19.975161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:19.975431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.031465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ea3c1-8a75-4c05-8f06-5b625f20d8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:20.032351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ea3c1-8a75-4c05-8f06-5b625f20d8d5 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.032549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ea3c1-8a75-4c05-8f06-5b625f20d8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.032683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ea3c1-8a75-4c05-8f06-5b625f20d8d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.033485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ea3c1-8a75-4c05-8f06-5b625f20d8d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:20.034873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ea3c1-8a75-4c05-8f06-5b625f20d8d5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:20.064756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 453/902] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:20.067579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7642-c0b8-444e-bc1a-9ad9a555effb None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.069042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7642-c0b8-444e-bc1a-9ad9a555effb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.069828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7642-c0b8-444e-bc1a-9ad9a555effb None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.072173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 454/903] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:20.077718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:20.079285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:20.080253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132c552-0967-41ac-a1a2-d2ebdf8a69e4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.080852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132c552-0967-41ac-a1a2-d2ebdf8a69e4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.081614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132c552-0967-41ac-a1a2-d2ebdf8a69e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.111519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7132c552-0967-41ac-a1a2-d2ebdf8a69e4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:20.114038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7132c552-0967-41ac-a1a2-d2ebdf8a69e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:20.233176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 455/904] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:20.247705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 450/905] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:19 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:50:20.248885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132c552-0967-41ac-a1a2-d2ebdf8a69e4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:20.261203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:20.280484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:20.287189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:20.405357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132c552-0967-41ac-a1a2-d2ebdf8a69e4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:20.453021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b86ef-663b-4c01-b3b9-eaa4442e30dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:20.453898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b86ef-663b-4c01-b3b9-eaa4442e30dd None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.454184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b86ef-663b-4c01-b3b9-eaa4442e30dd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.454463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b86ef-663b-4c01-b3b9-eaa4442e30dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.459327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380a476-88e3-4039-841e-d15ef6dbb3c1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:20.470194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b86ef-663b-4c01-b3b9-eaa4442e30dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:20.471704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b86ef-663b-4c01-b3b9-eaa4442e30dd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-8e6b86ef-663b-4c01-b3b9-eaa4442e30dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:20.516062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:20.517068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:20.517973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:20.517973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:20.517973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:20.517973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:20.518964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 456/906] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:20.521573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc57b6-daa4-4548-9d92-38489105052d None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.522060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc57b6-daa4-4548-9d92-38489105052d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.522537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc57b6-daa4-4548-9d92-38489105052d None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.525167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 457/907] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:20.530728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc57b6-daa4-4548-9d92-38489105052d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:20.540058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a4b54-6a79-4739-ad43-be07faf12b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:20.541041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a4b54-6a79-4739-ad43-be07faf12b14 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.541519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a4b54-6a79-4739-ad43-be07faf12b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.541973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a4b54-6a79-4739-ad43-be07faf12b14 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.542905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a4b54-6a79-4739-ad43-be07faf12b14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:20.544481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a4b54-6a79-4739-ad43-be07faf12b14 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:20.564025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 451/908] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:50:20 2022] POST /identity/v3/services => generated 241 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:20.566737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1ab1-ac3a-4b2d-a6f0-3189300ed60e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.567503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1ab1-ac3a-4b2d-a6f0-3189300ed60e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.568699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1ab1-ac3a-4b2d-a6f0-3189300ed60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.599683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-363c1ab1-ac3a-4b2d-a6f0-3189300ed60e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:20.602231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-363c1ab1-ac3a-4b2d-a6f0-3189300ed60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:20.753536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:20.771494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:20.776526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 452/909] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:20.781482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc57b6-daa4-4548-9d92-38489105052d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:20.820610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1ab1-ac3a-4b2d-a6f0-3189300ed60e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:20.824328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:20.825966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:20.825966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:20.825966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:20.826311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:20.827559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:20.829118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:20.843646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 458/910] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:20.849593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:20.992531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1ab1-ac3a-4b2d-a6f0-3189300ed60e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:21.002821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad0678-fbcb-4ee7-990b-d672b163b5c6 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:21.044332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:21.055823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:21.056226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:21.057120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:21.057545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:21.057984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:21.058093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:21.058741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:21.059325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:21.061113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:21.064403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:21.067072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:21.068919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:21.085091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:21.086134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:21.087014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:21.094269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:21.095216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:21.096044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:21.097388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 459/912] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:21.098492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:21.101302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 453/912] 10.222.0.43 () {56 vars in 1132 bytes} [Wed Jun 29 07:50:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:21.107861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:21.197432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:21.244959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e8b3-8878-4e24-bca0-632b80fe706d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:21.295690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb923f-cc94-49c6-abdf-efba0ea56f60 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:21.325119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:21.326199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:21.326644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:21.327066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:21.328620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:21.329122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:21.332620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:21.344866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:21.345959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:21.346452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:21.346928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:21.347935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:21.348514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:21.349941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 454/913] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:21.352272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:21.353687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:21.379322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 460/914] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:21.388249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:21.508030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:21.569574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be031c5-2d03-4ff9-8b4b-38c7429993f6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:21.574325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71417c3-62ac-4ca8-9c47-fbd5ab827a0e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:21.615035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:21.615726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:21.616007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:21.616284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:21.617221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] RBAC: Adding query filter params (role.id=2126aab4c72841deabc319457a5fab5c, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=c813112306f840008f94dec2d2c844f4) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:21.617579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:21.619041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:21.626483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 461/915] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:21 2022] GET /identity/v3/role_assignments?user.id=c813112306f840008f94dec2d2c844f4&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=2126aab4c72841deabc319457a5fab5c => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:21.637586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:21.649049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:21.649832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:21.650174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:21.650459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:21.651192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:21.651504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:21.652959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:21.659592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 455/916] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:21.669783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:21.885505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987d4d2-527b-4188-ac44-45c8d67eec58 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:21.921417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:21.955151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27201173-4bf0-4e38-8c5f-5554714311d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:21.957537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27201173-4bf0-4e38-8c5f-5554714311d2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:21.958338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27201173-4bf0-4e38-8c5f-5554714311d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:21.959114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27201173-4bf0-4e38-8c5f-5554714311d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:21.960897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27201173-4bf0-4e38-8c5f-5554714311d2 None admin] RBAC: Adding query filter params (interface=public, service_id=e8bbe37a9dd64e4ebabd22e150eecb0a, region_id=RegionOne) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:21.961775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27201173-4bf0-4e38-8c5f-5554714311d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:21.964626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27201173-4bf0-4e38-8c5f-5554714311d2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:21.976280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 462/917] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:50:21 2022] GET /identity/v3/endpoints?service_id=e8bbe37a9dd64e4ebabd22e150eecb0a&interface=public®ion_id=RegionOne => generated 194 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:21.984703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:22.146218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1342f-49f1-42c2-964a-e1214b41cd54 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:22.194185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70ab25-cab0-401f-8fa5-c044dfd45b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:22.195327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70ab25-cab0-401f-8fa5-c044dfd45b18 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:22.195825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70ab25-cab0-401f-8fa5-c044dfd45b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:22.196315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70ab25-cab0-401f-8fa5-c044dfd45b18 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:22.197646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70ab25-cab0-401f-8fa5-c044dfd45b18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:22.200216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70ab25-cab0-401f-8fa5-c044dfd45b18 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:22.213288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 456/918] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:50:21 2022] GET /identity/v3/services => generated 1497 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:50:22.609253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6dac5-f4c1-4dd5-a933-4bf58bf06d53 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:22.609543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6dac5-f4c1-4dd5-a933-4bf58bf06d53 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:22.609909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6dac5-f4c1-4dd5-a933-4bf58bf06d53 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:22.611633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 463/919] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:22.618379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3ef9b-f949-4ae2-ac81-b4c3fe253ed0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:22.619135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3ef9b-f949-4ae2-ac81-b4c3fe253ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:22.619135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3ef9b-f949-4ae2-ac81-b4c3fe253ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:22.639534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e7d3ef9b-f949-4ae2-ac81-b4c3fe253ed0 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:22.640111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e7d3ef9b-f949-4ae2-ac81-b4c3fe253ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:22.712071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:22.751487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 457/920] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:22.791903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6dac5-f4c1-4dd5-a933-4bf58bf06d53 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:23.043667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6dac5-f4c1-4dd5-a933-4bf58bf06d53 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:23.105497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c8054-614a-4e98-9d10-b50e3269e754 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.105984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c8054-614a-4e98-9d10-b50e3269e754 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.107021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c8054-614a-4e98-9d10-b50e3269e754 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.113011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 458/921] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:23.117491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab5e6-61d8-49ba-82d4-1c2d76b71175 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.117884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:23.118378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab5e6-61d8-49ba-82d4-1c2d76b71175 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.118865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab5e6-61d8-49ba-82d4-1c2d76b71175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.119393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.119873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.120385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.121806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:23.124541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:23.128957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING keystone.server.flask.common [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 07:50:23.143049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2adab5e6-61d8-49ba-82d4-1c2d76b71175 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:23.144952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2adab5e6-61d8-49ba-82d4-1c2d76b71175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:23.168054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.sql.core [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 07:50:23.168054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 07:50:23.168054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [parameters: {'id': '886ec6f1452e4e0b9de9261964b41714', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 07:50:23.168054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80108) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 886ec6f1452e4e0b9de9261964b41714.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 886ec6f1452e4e0b9de9261964b41714. Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application conn.query(q) Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 07:50:23.191868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application result.read() Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 29 07:50:23.192845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application next(self.gen) Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application yield resource Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application next(self.gen) Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise self.value Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application session.commit() Jun 29 07:50:23.194860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self.session.flush() Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application compat.raise_( Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 07:50:23.195930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 07:50:23.197033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application conn.query(q) Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application result.read() Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 07:50:23.197856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [parameters: {'id': '886ec6f1452e4e0b9de9261964b41714', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:23.198721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:23.199523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 886ec6f1452e4e0b9de9261964b41714. Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.200483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 464/922] 10.222.0.43 () {60 vars in 1140 bytes} [Wed Jun 29 07:50:22 2022] POST /identity/v3/projects => generated 262 bytes in 409 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:23.210797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:23.261771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 459/923] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:23.278544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:23.289174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adab5e6-61d8-49ba-82d4-1c2d76b71175 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:23.417643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adab5e6-61d8-49ba-82d4-1c2d76b71175 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:23.433377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482c7aa-ff02-4ca3-a9cb-324a0a9f7281 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:23.439260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:23.474801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:23.477571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.478896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.479681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.481639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:23.484835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:23.503958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:23.505002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:23.507166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:23.517410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 460/924] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:23.518748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:23.520269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.521515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.522321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.522506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:23.535830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:23.539956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:23.568844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:23.570968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:23.573054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:23.581358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 465/925] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:23.586614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:23.680375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c4aaf-13f5-481b-8d96-b2c87f0f6294 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:23.757237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:23.758605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.759127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.759722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.761253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:23.761816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:23.764800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:23.780964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 461/926] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:23.782792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:23.798935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:23.821885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fff921-8d97-45dc-97e9-8170792e6e9f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:23.854533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:23.905654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191eb864-3e80-4768-bfdc-b17df060ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:23.907694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eb864-3e80-4768-bfdc-b17df060ff1d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.908189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eb864-3e80-4768-bfdc-b17df060ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.908669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eb864-3e80-4768-bfdc-b17df060ff1d None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.910489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191eb864-3e80-4768-bfdc-b17df060ff1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:23.911783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191eb864-3e80-4768-bfdc-b17df060ff1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:23.915159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191eb864-3e80-4768-bfdc-b17df060ff1d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:23.949208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b37006-96ea-492d-9920-859c529a9473 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:23.950900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 466/927] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:23.982928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762aaf1-c048-4f8f-a045-7f836f6cc1c3 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.983274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762aaf1-c048-4f8f-a045-7f836f6cc1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.983489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762aaf1-c048-4f8f-a045-7f836f6cc1c3 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:23.985214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 467/928] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:23.994112 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72536c-9eb3-44ef-b33a-e95c50caf42c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:23.994601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72536c-9eb3-44ef-b33a-e95c50caf42c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:23.995457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72536c-9eb3-44ef-b33a-e95c50caf42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:24.017603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9b72536c-9eb3-44ef-b33a-e95c50caf42c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:24.019246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9b72536c-9eb3-44ef-b33a-e95c50caf42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:24.028531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:24.029267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:24.029554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:24.029835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:24.030560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:24.030874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:24.032301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:24.039094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 462/929] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:24.045622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:24.136086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 468/930] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:24.170176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72536c-9eb3-44ef-b33a-e95c50caf42c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:24.242545 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39361c28-00c9-49c4-a35d-2d5cadee63e5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:24.251687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:24.292028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:24.292757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:24.293192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:24.293374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:24.294169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] RBAC: Adding query filter params (interface=public, service_id=5503b93c4b95494ba1b4db7e72950a7d, region_id=RegionOne) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:24.294481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:24.295898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:24.302895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 463/931] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:50:24 2022] GET /identity/v3/endpoints?service_id=5503b93c4b95494ba1b4db7e72950a7d&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:24.309324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:24.340275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:24.369250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72536c-9eb3-44ef-b33a-e95c50caf42c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:24.438630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:24.439495 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:24.439782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:24.440321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:24.447435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75912a-2c0d-462e-9e0d-4a31df89b32a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:24.456477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:24.467639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:24.468662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:24.469769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 469/932] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jun 29 07:50:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:24.477708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:24.528491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:24.529619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:24.530129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:24.530612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:24.532241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:24.534948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:24.557019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 464/933] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:50:24 2022] GET /identity/v3/services => generated 1497 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 07:50:24.680434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18581-9208-4b7c-9a6d-50b0909d541a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:24.729467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:24.730501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:24.730962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:24.731391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:24.732414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:24.732883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:24.734491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:24.750812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 470/934] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jun 29 07:50:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:24.753263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:24.805236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:24.892682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f0ff0-4997-4199-8df4-8de9d2877e2d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:24.940472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78114-435b-4af6-aee9-3aa3665355c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:24.941725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78114-435b-4af6-aee9-3aa3665355c6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:24.942157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78114-435b-4af6-aee9-3aa3665355c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:24.942634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78114-435b-4af6-aee9-3aa3665355c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:24.959330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f78114-435b-4af6-aee9-3aa3665355c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:24.961601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f78114-435b-4af6-aee9-3aa3665355c6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-21f78114-435b-4af6-aee9-3aa3665355c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:24.986202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:24.987906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:24.990183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:24.990183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:24.990183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:24.990183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:24.990183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 465/935] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:24.994301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:24.998357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:25.011794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa818ab0-a88f-4a3f-98be-3812d74fb1f4 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.012123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa818ab0-a88f-4a3f-98be-3812d74fb1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.012361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa818ab0-a88f-4a3f-98be-3812d74fb1f4 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.014058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 466/936] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:25.019368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365881-89f4-46dd-958c-d31c64a07a53 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.019656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365881-89f4-46dd-958c-d31c64a07a53 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.019933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365881-89f4-46dd-958c-d31c64a07a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.041254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c8365881-89f4-46dd-958c-d31c64a07a53 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:25.042540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c8365881-89f4-46dd-958c-d31c64a07a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:25.117951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df740632-ba5e-426c-9e9c-23df231a5b97 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:25.133623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 467/937] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:25.156219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8365881-89f4-46dd-958c-d31c64a07a53 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:25.157935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:25.159248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.159749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.160251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.161779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:25.162306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:25.163946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:25.177693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:25.183579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 471/938] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:25.185415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:25.256269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:25.316862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8365881-89f4-46dd-958c-d31c64a07a53 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:25.338387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40e835-e381-4107-81f0-96321e923c03 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:25.361875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:25.362831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.363182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.363529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.364272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:25.366307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:25.379377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:25.381145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 07:50:25.382286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:25.388050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 468/939] 10.222.0.43 () {56 vars in 1132 bytes} [Wed Jun 29 07:50:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:25.389057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:25.407039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:25.408469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.409170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.409659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.425589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:25.428171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:25.462071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:25.464367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:25.465964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 472/940] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:50:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:25.473111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:25.479012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:25.501401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7e9a6-ccf7-490a-bc6a-e0bf189936e2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:25.539269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:25.539953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.540239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.540521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.541276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:25.541589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:25.542995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:25.557378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 469/941] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:25.560414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:25.644154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e506d-2ddd-4eb8-ae68-fa29f1d39b5a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:25.676792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:25.691991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:25.693170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.693610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.694038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.695071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:25.695540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:25.697220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:25.730809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bd33f-3d62-4c7b-aa51-e55df8c42d4d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:25.744334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 473/942] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jun 29 07:50:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:25.746336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:25.805487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:25.806958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.807448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.807919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.809006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:25.809610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:25.811605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:25.825319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 470/943] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:25.830257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:25.848348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:25.901226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f6190-87f8-4561-9f06-a696f5274f4d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:25.947946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:25.948763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:25.949154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:25.949448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:25.963611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:25.965187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:25.990905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:25.991871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:25.993642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:25.993642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:25.993642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:25.993642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:25.993642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 474/944] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:26.000554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:26.017573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ddd63-263d-4d51-b4ab-4389d7ff1713 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:26.080755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:26.093433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:26.094673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:26.095060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:26.095627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:26.096724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:26.100328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:26.158644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 471/945] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:26.202727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2382-770d-48b3-890e-0615ed665fbd None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:26.274403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:26.275524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:26.276038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:26.276535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:26.277583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:26.278137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:26.279759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:26.295139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 475/946] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:26.304397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:26.387703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:26.449715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360ece-38d8-4257-9a76-b70a0de8ae27 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:26.519666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761d13e-b94a-411f-be99-e0565cbc928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:26.521113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761d13e-b94a-411f-be99-e0565cbc928d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:26.521412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761d13e-b94a-411f-be99-e0565cbc928d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:26.521861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761d13e-b94a-411f-be99-e0565cbc928d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:26.534804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761d13e-b94a-411f-be99-e0565cbc928d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:26.537750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761d13e-b94a-411f-be99-e0565cbc928d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-8761d13e-b94a-411f-be99-e0565cbc928d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:26.563361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:26.564349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:26.565314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:26.573168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 472/947] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:26.575708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:26.702174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99d7e-ff65-4741-bdba-cfecc3108de2 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:26.702591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99d7e-ff65-4741-bdba-cfecc3108de2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:26.703127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99d7e-ff65-4741-bdba-cfecc3108de2 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:26.705374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:26.705744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 473/948] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:26.712355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ce008-0fe2-4b49-963d-26529c182ee1 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:26.712747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ce008-0fe2-4b49-963d-26529c182ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:26.713050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ce008-0fe2-4b49-963d-26529c182ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:26.725424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cec33-4f08-45a6-967c-4ae387ab3ca3 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:26.733157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bf4ce008-0fe2-4b49-963d-26529c182ee1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:26.734408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bf4ce008-0fe2-4b49-963d-26529c182ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:26.776600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:26.777653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:26.778132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:26.778620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:26.779679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:26.780225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:26.781984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:26.798096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 476/949] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:26.806066 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:26.841125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 474/950] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:26.867602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ce008-0fe2-4b49-963d-26529c182ee1 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:26.933035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:26.995350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ce008-0fe2-4b49-963d-26529c182ee1 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:27.001556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55496070-b268-44b5-8088-a7d340172b48 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:27.061769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:27.062930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.063415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.063902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.065251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:27.068108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:27.070530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING keystone.server.flask.common [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 07:50:27.077427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220a667-bfea-44c3-88a4-ec85da95be19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:27.079529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a667-bfea-44c3-88a4-ec85da95be19 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.080364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a667-bfea-44c3-88a4-ec85da95be19 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.081155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a667-bfea-44c3-88a4-ec85da95be19 None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/c813112306f840008f94dec2d2c844f4/roles/2126aab4c72841deabc319457a5fab5c` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.086198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220a667-bfea-44c3-88a4-ec85da95be19 None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=c813112306f840008f94dec2d2c844f4, role_id=2126aab4c72841deabc319457a5fab5c)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:27.089759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220a667-bfea-44c3-88a4-ec85da95be19 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:27.094011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.sql.core [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 07:50:27.094011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 07:50:27.094011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [parameters: {'user_id': 'bbcffbecab884540b5a30fe47ac05ec5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 07:50:27.094011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80107) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-9bbd4d61-f747-47c7-8c0d-1b8ede71538e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application conn.query(q) Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result.read() Jun 29 07:50:27.111762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 07:50:27.113967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application next(self.gen) Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application yield resource Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application next(self.gen) Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise self.value Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application session.commit() Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 07:50:27.116358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.session.flush() Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application compat.raise_( Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 07:50:27.119080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 07:50:27.121199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application conn.query(q) Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application result.read() Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 07:50:27.123321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bbcffbecab884540b5a30fe47ac05ec5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:27.127811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 29 07:50:27.130333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 475/951] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jun 29 07:50:26 2022] POST /identity/v3/users => generated 160 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d915a-b594-4ed7-933e-b3eecf489043 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d915a-b594-4ed7-933e-b3eecf489043 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.140041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d915a-b594-4ed7-933e-b3eecf489043 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.142261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 476/952] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:27 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:27.142261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d915a-b594-4ed7-933e-b3eecf489043 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:27.142554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 477/953] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:50:26 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/c813112306f840008f94dec2d2c844f4/roles/2126aab4c72841deabc319457a5fab5c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:50:27.143489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4856b6-f394-4e9a-b10d-f8e044a680ae None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.144111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4856b6-f394-4e9a-b10d-f8e044a680ae None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.144616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4856b6-f394-4e9a-b10d-f8e044a680ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.185811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-5a4856b6-f394-4e9a-b10d-f8e044a680ae None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:27.188790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5a4856b6-f394-4e9a-b10d-f8e044a680ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:27.206696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:27.354173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d915a-b594-4ed7-933e-b3eecf489043 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:27.358865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 478/954] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:27.386479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4856b6-f394-4e9a-b10d-f8e044a680ae None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:27.397379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:27.424225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:27.425108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.426430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.426734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.438063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:27.439889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:27.461509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:27.462518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 07:50:27.463524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:27.464090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 477/955] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:27.473399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:27.558065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4856b6-f394-4e9a-b10d-f8e044a680ae None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:27.600123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:27.601514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.601888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.602243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.603031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:27.604656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:27.620945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:27.623354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:27.626151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 479/956] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:27.629897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:27.638397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:27.676776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a124fa3-8b24-42b9-9f84-15771dc2ba60 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:27.718450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:27.719148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.719427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.719707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.720547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:27.720862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:27.722544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:27.769444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 478/957] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jun 29 07:50:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:27.838582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a9de1-589b-485f-9cdf-e76df8aa4ecc None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:27.904694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:27.906405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:27.907254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:27.908051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:27.909777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:27.910667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:27.913813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:27.924957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 480/958] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:27.934221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:27.990168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:28.105791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392f1a6-03c9-4719-8e05-38444ce42c90 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:28.156919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8d90e-4db3-4b5d-8204-92dc1ca37079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:28.157969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8d90e-4db3-4b5d-8204-92dc1ca37079 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:28.158479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8d90e-4db3-4b5d-8204-92dc1ca37079 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:28.158960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8d90e-4db3-4b5d-8204-92dc1ca37079 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:28.159876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8d90e-4db3-4b5d-8204-92dc1ca37079 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:28.160403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8d90e-4db3-4b5d-8204-92dc1ca37079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:28.162843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8d90e-4db3-4b5d-8204-92dc1ca37079 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:28.169069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 479/959] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:28.178373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:28.256278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:28.299050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352320a-9a6d-437d-bce2-5fb2bd9a3c33 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:28.339671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4213-bfdc-4f0e-9630-61369fdadfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:28.340880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4213-bfdc-4f0e-9630-61369fdadfec None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:28.341482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4213-bfdc-4f0e-9630-61369fdadfec None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:28.341997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4213-bfdc-4f0e-9630-61369fdadfec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:28.343504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc4213-bfdc-4f0e-9630-61369fdadfec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:28.346207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc4213-bfdc-4f0e-9630-61369fdadfec None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:28.389061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 481/960] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:29.033007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b5ae0-37df-4fa7-a8f4-29e316eb06e9 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.033597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b5ae0-37df-4fa7-a8f4-29e316eb06e9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.033597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b5ae0-37df-4fa7-a8f4-29e316eb06e9 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.035366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 480/961] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:29.041867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d1354-d006-4b92-956b-ebdcf04fa6ff None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.042176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d1354-d006-4b92-956b-ebdcf04fa6ff None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.042524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d1354-d006-4b92-956b-ebdcf04fa6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.061464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-eb2d1354-d006-4b92-956b-ebdcf04fa6ff None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:29.062743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-eb2d1354-d006-4b92-956b-ebdcf04fa6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:29.158229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 482/962] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:29.182524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b5ae0-37df-4fa7-a8f4-29e316eb06e9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:29.367891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:29.379935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b5ae0-37df-4fa7-a8f4-29e316eb06e9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:29.430749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d98990-7cc6-4a15-ac2c-c80f18789164 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.431523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d98990-7cc6-4a15-ac2c-c80f18789164 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.431523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d98990-7cc6-4a15-ac2c-c80f18789164 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.433036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 483/963] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:29.438269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb0007-cd01-4a70-b0c1-ed88fab369a3 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.438537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb0007-cd01-4a70-b0c1-ed88fab369a3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.438836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb0007-cd01-4a70-b0c1-ed88fab369a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.448013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:29.449412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.449918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.450417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.451419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:29.454038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:29.457918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-64eb0007-cd01-4a70-b0c1-ed88fab369a3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:29.459489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-64eb0007-cd01-4a70-b0c1-ed88fab369a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:29.477137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:29.478534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:29.479555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:29.486516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 481/964] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:29.487936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:29.543135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:29.581125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 484/965] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:29.622005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb0007-cd01-4a70-b0c1-ed88fab369a3 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:29.649193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525313d2-f3e2-4a0b-b697-7999e4c93054 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:29.694896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:29.695620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.695918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.696172 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.697068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:29.697373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:29.698770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:29.705247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 482/966] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:29.713573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:29.757712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:29.797372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb0007-cd01-4a70-b0c1-ed88fab369a3 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:29.851130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:29.852442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.852908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.853428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.872846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:29.887659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:29.889988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:29.892045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 485/967] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jun 29 07:50:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:29.897646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:29.899387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d75cfc-6410-4e0a-9e64-afda644e8dc6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:29.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:29.955094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:29.955375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:29.955655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:29.956390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:29.957000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:29.959638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:29.966776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 483/968] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:29.977915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:30.070712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd99a6-554b-48fa-8d19-986da4575eb3 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:30.116080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:30.116778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:30.117098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:30.117384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:30.118330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:30.118491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:30.119873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:30.130686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 486/969] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jun 29 07:50:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:30.137732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:30.309148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:30.329391 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1106-98a9-41ee-8014-82d46e8de87a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:30.396745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98c63d-32aa-4d7d-bc52-e8d72dd3b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:30.398158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c63d-32aa-4d7d-bc52-e8d72dd3b17b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:30.398663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c63d-32aa-4d7d-bc52-e8d72dd3b17b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:30.399152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c63d-32aa-4d7d-bc52-e8d72dd3b17b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:30.421450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98c63d-32aa-4d7d-bc52-e8d72dd3b17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:30.424190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98c63d-32aa-4d7d-bc52-e8d72dd3b17b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-7a98c63d-32aa-4d7d-bc52-e8d72dd3b17b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:30.458527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:30.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:30.462225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:30.462225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:30.462225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:30.462225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:30.465092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 484/970] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:30.470113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:30.658318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e8d8a-a859-418b-8e1b-cbdec9896f28 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:30.658582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e8d8a-a859-418b-8e1b-cbdec9896f28 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:30.658854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e8d8a-a859-418b-8e1b-cbdec9896f28 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:30.663795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 485/971] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:30.669556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571260b4-4843-4e74-a569-e79cd8b89a50 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:30.670025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571260b4-4843-4e74-a569-e79cd8b89a50 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:30.670406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571260b4-4843-4e74-a569-e79cd8b89a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:30.682639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfaea-e8a7-4c51-b4cc-8aa7eb8b11aa None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:30.686386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:30.695075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-571260b4-4843-4e74-a569-e79cd8b89a50 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:30.697876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-571260b4-4843-4e74-a569-e79cd8b89a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:30.748722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:30.750022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:30.750497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:30.750952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:30.752258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:30.752815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:30.755982 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:30.783161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 487/972] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:30.785513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:30.833082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 486/973] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:30.854280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571260b4-4843-4e74-a569-e79cd8b89a50 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:30.957136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:30.958696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542a67-3fb0-4709-8791-a386acb8969e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:31.002030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:31.002975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:31.003267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:31.003540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:31.013484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571260b4-4843-4e74-a569-e79cd8b89a50 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:31.013765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:31.015079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:31.033228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:31.035396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 07:50:31.039030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:31.040399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 488/974] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:50:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:31.040931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:31.079212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:31.080045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:31.080369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:31.081579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:31.081756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:31.081930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:31.082775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:31.108636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:31.109768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:31.110612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:31.114959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 487/975] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:31.117951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:31.216905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c278-7f74-4c16-b350-71ffb2ba49c5 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:31.261205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:31.261903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:31.262198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:31.262469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:31.263339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:31.263656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:31.265167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:31.291454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 489/976] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jun 29 07:50:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:31.300681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:31.322888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bc704-c552-4209-af2c-dc903246d3a7 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:31.377461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:31.378611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:31.379130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:31.379639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:31.382133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:31.382559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:31.385228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:31.388614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:31.405064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 488/977] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:31.406743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:31.424534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:31.450354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c935041-b7f0-4c26-b361-6808a35346a8 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:31.503576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:31.504611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:31.504611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:31.505098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:31.580344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551439d-751f-4f10-ad36-27ca0b3c3ddd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:31.620706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:31.622201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:31.644138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:31.646027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:31.646541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:31.647058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:31.648535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:31.649243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:31.651571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:31.667590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 489/978] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:31.669431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:31.738248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:31.749659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:31.776413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 490/979] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:31 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 07:50:31.784925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:31.806123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21350bc-0f1a-455a-8cf6-68ea8855bc48 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:31.855876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:31.857131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:31.857619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:31.858072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:31.859694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] RBAC: Adding query filter params (interface=admin, service_id=5503b93c4b95494ba1b4db7e72950a7d, region_id=RegionOne) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:31.860238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:31.863705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:31.877043 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 490/980] 10.222.0.43 () {56 vars in 1249 bytes} [Wed Jun 29 07:50:31 2022] GET /identity/v3/endpoints?service_id=5503b93c4b95494ba1b4db7e72950a7d&interface=admin®ion_id=RegionOne => generated 193 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:31.881969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:31.987335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e4d2-5530-4f54-b9a7-48614fc31bb3 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:32.053404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:32.054808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.055340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.055838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.062459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95676752-0ec3-4ba9-a34c-970aec362d95 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:32.073486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:32.075094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:32.094253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:32.096152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:32.098405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:32.098405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:32.098405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:32.098405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:32.100310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 491/981] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:32.101281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:32.111213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:32.121016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152aa58-19b5-4538-b0a2-a057b11b0137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:32.121696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152aa58-19b5-4538-b0a2-a057b11b0137 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.121985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152aa58-19b5-4538-b0a2-a057b11b0137 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.122265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152aa58-19b5-4538-b0a2-a057b11b0137 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.122966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152aa58-19b5-4538-b0a2-a057b11b0137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:32.124367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152aa58-19b5-4538-b0a2-a057b11b0137 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:32.131185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 491/982] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:50:31 2022] GET /identity/v3/services => generated 1497 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:50:32.164128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:32.311314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46713b-1a61-499e-a4f2-d1d406e69c2c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:32.375288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:32.375970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.376281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.376552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.377295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:32.377601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:32.378985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:32.386897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776c76-2d80-4c4f-b459-9848322e60d6 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.387640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776c76-2d80-4c4f-b459-9848322e60d6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.387640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776c76-2d80-4c4f-b459-9848322e60d6 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.388861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 492/983] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:32.394348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0267a-9803-4eac-8cee-f93484540f14 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.394712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0267a-9803-4eac-8cee-f93484540f14 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.394884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 492/984] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:32.395066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0267a-9803-4eac-8cee-f93484540f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.404535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:32.413794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-39b0267a-9803-4eac-8cee-f93484540f14 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:32.414999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-39b0267a-9803-4eac-8cee-f93484540f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:32.500571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 493/985] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:32.525503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0267a-9803-4eac-8cee-f93484540f14 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:32.543135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:32.556460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a4bb-0485-44ac-9d49-0f9446d5f427 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:32.600025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:32.601154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.601629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.601899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.609649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:32.611224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:32.627924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:32.629011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:32.630524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 493/986] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:32.635478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:32.659529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0267a-9803-4eac-8cee-f93484540f14 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:32.691052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:32.708052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:32.708844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.709240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.709651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.710269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:32.711661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:32.733101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 494/987] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:50:32 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:32.760610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73137b-5de2-4dd8-be85-ce5df7e01e35 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:32.812835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88191d9c-add8-4097-a235-53ebe4cb8a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:32.813573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191d9c-add8-4097-a235-53ebe4cb8a3f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:32.813863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191d9c-add8-4097-a235-53ebe4cb8a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:32.814145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191d9c-add8-4097-a235-53ebe4cb8a3f None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:32.814971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191d9c-add8-4097-a235-53ebe4cb8a3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:32.815289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191d9c-add8-4097-a235-53ebe4cb8a3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:32.816789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191d9c-add8-4097-a235-53ebe4cb8a3f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:32.837018 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 494/988] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:32.839269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:32.965518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8bf31-fe54-45cd-9893-660ea30117c0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:33.027522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85c38a-dfe7-4ac2-b4f1-dfc638ba07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:33.028730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85c38a-dfe7-4ac2-b4f1-dfc638ba07aa None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:33.029282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85c38a-dfe7-4ac2-b4f1-dfc638ba07aa None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:33.029763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85c38a-dfe7-4ac2-b4f1-dfc638ba07aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:33.031550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85c38a-dfe7-4ac2-b4f1-dfc638ba07aa None admin] RBAC: Adding query filter params (role.id=2126aab4c72841deabc319457a5fab5c, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=c813112306f840008f94dec2d2c844f4) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:33.032123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85c38a-dfe7-4ac2-b4f1-dfc638ba07aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:33.035064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85c38a-dfe7-4ac2-b4f1-dfc638ba07aa None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:33.047960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 495/989] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:32 2022] GET /identity/v3/role_assignments?user.id=c813112306f840008f94dec2d2c844f4&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=2126aab4c72841deabc319457a5fab5c => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:34.630698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129610c-cdcb-47a7-8245-bdaf42474a11 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:34.631244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129610c-cdcb-47a7-8245-bdaf42474a11 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:34.631666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129610c-cdcb-47a7-8245-bdaf42474a11 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:34.634319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 495/990] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:34.639966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355927-213d-4bcf-b885-3e72b5b10bed None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:34.640660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355927-213d-4bcf-b885-3e72b5b10bed None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:34.640660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355927-213d-4bcf-b885-3e72b5b10bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:34.659232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-19355927-213d-4bcf-b885-3e72b5b10bed None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:34.660440 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-19355927-213d-4bcf-b885-3e72b5b10bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:34.715177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:34.761318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 496/991] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:34.785413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129610c-cdcb-47a7-8245-bdaf42474a11 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:34.805298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:34.964663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c0e5d-21d6-4f8c-8eca-c17539377110 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:34.965012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c0e5d-21d6-4f8c-8eca-c17539377110 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:34.965280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c0e5d-21d6-4f8c-8eca-c17539377110 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:34.966818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 497/992] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:34.968427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129610c-cdcb-47a7-8245-bdaf42474a11 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:34.971878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5ad54-e86a-42ba-bf5d-f1e793483ea4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:34.972175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5ad54-e86a-42ba-bf5d-f1e793483ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:34.972463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5ad54-e86a-42ba-bf5d-f1e793483ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:34.990809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-73d5ad54-e86a-42ba-bf5d-f1e793483ea4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:34.992025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-73d5ad54-e86a-42ba-bf5d-f1e793483ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:35.023050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:35.024188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.024687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.025192 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.026275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:35.027826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:35.040840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:35.042086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 07:50:35.043119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:35.050945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 496/993] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jun 29 07:50:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:35.052184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:35.084765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 498/994] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:35.107919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5ad54-e86a-42ba-bf5d-f1e793483ea4 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:35.151164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:35.206027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945feccf-16fd-4dca-87ed-da68a8a83936 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:35.274901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:35.275515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5ad54-e86a-42ba-bf5d-f1e793483ea4 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:35.275915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.276409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.276890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.279611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:35.280164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:35.282766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:35.284052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:35.294439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 497/995] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:35.303611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:35.331062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:35.332426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.333461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.334024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.335276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:35.337949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:35.360920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:35.362160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:35.363110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:35.365052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 499/996] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:35.371234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:35.470054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7ec9-0f1a-4f83-b287-6638db25250d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:35.500729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e7bdc-ff09-4f8a-9ec4-85371b105eb5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:35.544421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:35.545493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.545965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.546425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.547392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:35.547885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:35.548414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:35.548910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.549465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.549982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.550832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:35.551300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:35.552478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:35.554794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:35.555620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:35.561693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 498/997] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jun 29 07:50:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:35.565044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 500/998] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:35.570941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:35.571496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:35.624113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:35.716648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e4192-98a5-4bbb-89d6-40b7477291a9 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:35.735637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a887b-c891-4f92-aee3-8e5975ebd1f1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:35.772268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e424dfb-63b5-4dff-83f2-12db2612b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:35.773464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424dfb-63b5-4dff-83f2-12db2612b01b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.773980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424dfb-63b5-4dff-83f2-12db2612b01b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.774491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424dfb-63b5-4dff-83f2-12db2612b01b None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.775841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e424dfb-63b5-4dff-83f2-12db2612b01b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:35.776391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e424dfb-63b5-4dff-83f2-12db2612b01b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:35.779001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e424dfb-63b5-4dff-83f2-12db2612b01b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:35.779622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76beb3-d15a-4cb0-8719-336c932dfbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:35.780347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76beb3-d15a-4cb0-8719-336c932dfbd3 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.780696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76beb3-d15a-4cb0-8719-336c932dfbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.781234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76beb3-d15a-4cb0-8719-336c932dfbd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.781902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76beb3-d15a-4cb0-8719-336c932dfbd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:35.783407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76beb3-d15a-4cb0-8719-336c932dfbd3 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:35.787931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 499/999] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:35.790361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff7e95-76a5-481d-9ee3-4f3afbe96522 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.790874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff7e95-76a5-481d-9ee3-4f3afbe96522 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.791349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff7e95-76a5-481d-9ee3-4f3afbe96522 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.793948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 500/1000] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:35.795376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff7e95-76a5-481d-9ee3-4f3afbe96522 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:35.821143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 501/1001] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:35.821143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377f06-c8d0-453f-8a4d-f14197c5295d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:35.821143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377f06-c8d0-453f-8a4d-f14197c5295d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:35.821143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377f06-c8d0-453f-8a4d-f14197c5295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:35.839911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e6377f06-c8d0-453f-8a4d-f14197c5295d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:35.841291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e6377f06-c8d0-453f-8a4d-f14197c5295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:35.909556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:35.969273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 502/1002] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:35.980458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff7e95-76a5-481d-9ee3-4f3afbe96522 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:35.999196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6377f06-c8d0-453f-8a4d-f14197c5295d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:36.036076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:36.046125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:36.046536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:36.046924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:36.047396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:36.049156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] RBAC: Adding query filter params (interface=public, service_id=5f11ffa7044a4548ad89d285adde7d81, region_id=RegionOne) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:36.049610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:36.053119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:36.068153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 501/1003] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:50:35 2022] GET /identity/v3/endpoints?service_id=5f11ffa7044a4548ad89d285adde7d81&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:36.074258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:36.187085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6377f06-c8d0-453f-8a4d-f14197c5295d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:36.223099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4539359-02a1-447c-8055-8f8baf91e113 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:36.234508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:36.235433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:36.235877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:36.236269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:36.236939 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:36.238485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:36.253954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:36.256287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:36.257173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 503/1004] 10.222.0.43 () {56 vars in 1105 bytes} [Wed Jun 29 07:50:35 2022] GET /identity/v3/services/image => generated 86 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:36.264527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:36.271869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:36.286285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:36.287374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:36.287994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:36.288350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:36.289690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:36.292291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:36.305506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 502/1005] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:50:36 2022] GET /identity/v3/services => generated 1727 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 07:50:36.365833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:36.386190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6bcf6-67aa-4742-9138-18d6469cbcc1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:36.426946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35287bcb-0752-43df-8e3e-f6148518b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:36.427985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35287bcb-0752-43df-8e3e-f6148518b54d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:36.428456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35287bcb-0752-43df-8e3e-f6148518b54d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:36.428910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35287bcb-0752-43df-8e3e-f6148518b54d None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:36.429868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35287bcb-0752-43df-8e3e-f6148518b54d None admin] RBAC: Adding query filter params (name=image) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:36.430393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35287bcb-0752-43df-8e3e-f6148518b54d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:36.431998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35287bcb-0752-43df-8e3e-f6148518b54d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:36.443300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 504/1006] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:36.445088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:36.607079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02937c-5fbe-4031-a03d-659d3f85fa67 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:36.649267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50c878-6c2e-4d32-846b-ecbb6d56bd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:36.650256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50c878-6c2e-4d32-846b-ecbb6d56bd1c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:36.650715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50c878-6c2e-4d32-846b-ecbb6d56bd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:36.651161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50c878-6c2e-4d32-846b-ecbb6d56bd1c None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:36.652089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50c878-6c2e-4d32-846b-ecbb6d56bd1c None admin] RBAC: Adding query filter params (type=image) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:36.652571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50c878-6c2e-4d32-846b-ecbb6d56bd1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:36.654162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50c878-6c2e-4d32-846b-ecbb6d56bd1c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:36.659981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 503/1007] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:38.656881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2919c2-53cf-4a0c-96d9-1362bc363e3a None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:38.657707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2919c2-53cf-4a0c-96d9-1362bc363e3a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:38.657843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2919c2-53cf-4a0c-96d9-1362bc363e3a None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:38.659603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 505/1008] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:38.666509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faec0a0-17b0-4a42-b0e8-563e4ecd98ff None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:38.667337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faec0a0-17b0-4a42-b0e8-563e4ecd98ff None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:38.667476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faec0a0-17b0-4a42-b0e8-563e4ecd98ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:38.687832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6faec0a0-17b0-4a42-b0e8-563e4ecd98ff None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:38.689119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6faec0a0-17b0-4a42-b0e8-563e4ecd98ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:38.755187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:38.782788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 504/1009] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:38.807407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2919c2-53cf-4a0c-96d9-1362bc363e3a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:38.845791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38785d-0683-4c0f-b718-b272b7638f1f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:38.846612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38785d-0683-4c0f-b718-b272b7638f1f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:38.847090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38785d-0683-4c0f-b718-b272b7638f1f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:38.849845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 505/1010] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:38.858710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14cd6c-76f6-432b-bb4c-7fb8b9b733ec None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:38.859208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14cd6c-76f6-432b-bb4c-7fb8b9b733ec None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:38.859706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14cd6c-76f6-432b-bb4c-7fb8b9b733ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:38.888224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cf14cd6c-76f6-432b-bb4c-7fb8b9b733ec None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:38.890050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cf14cd6c-76f6-432b-bb4c-7fb8b9b733ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:38.973939 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2919c2-53cf-4a0c-96d9-1362bc363e3a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:39.006952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 506/1011] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:39.021713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:39.022530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.022819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.023074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.023730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:39.025120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:39.038809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:39.040672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 07:50:39.042851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:39.045035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 506/1012] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:39.046233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14cd6c-76f6-432b-bb4c-7fb8b9b733ec None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:39.049794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:39.078938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:39.200000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14cd6c-76f6-432b-bb4c-7fb8b9b733ec None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:39.200535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d11d1-dfdd-4a86-8ebe-fbadfeca2049 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:39.222421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:39.244213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333517bd-09d1-403f-ba9f-a9d9af935fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:39.245038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333517bd-09d1-403f-ba9f-a9d9af935fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.245433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333517bd-09d1-403f-ba9f-a9d9af935fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.245636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:39.245889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333517bd-09d1-403f-ba9f-a9d9af935fa2 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.246703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333517bd-09d1-403f-ba9f-a9d9af935fa2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:39.246941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.247190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333517bd-09d1-403f-ba9f-a9d9af935fa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:39.247428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.247900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.248833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333517bd-09d1-403f-ba9f-a9d9af935fa2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:39.249961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:39.252575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:39.255172 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 507/1013] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:39.263847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 507/1014] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:39.264057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac475efc-ce35-41da-a779-409b88bbbd15 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.264057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac475efc-ce35-41da-a779-409b88bbbd15 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.264057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac475efc-ce35-41da-a779-409b88bbbd15 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.264057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 508/1015] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:39.265962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:39.266285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac475efc-ce35-41da-a779-409b88bbbd15 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:39.382325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:39.404955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac475efc-ce35-41da-a779-409b88bbbd15 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:39.410702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb09b6b-eca8-427f-ad56-83e6de6fc49b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:39.444301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164ddbc-c441-403a-9e17-7e2434fca739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:39.445020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164ddbc-c441-403a-9e17-7e2434fca739 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.445310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164ddbc-c441-403a-9e17-7e2434fca739 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.445590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164ddbc-c441-403a-9e17-7e2434fca739 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.446325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164ddbc-c441-403a-9e17-7e2434fca739 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:39.446626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164ddbc-c441-403a-9e17-7e2434fca739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:39.448015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164ddbc-c441-403a-9e17-7e2434fca739 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:39.453410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 509/1016] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jun 29 07:50:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:39.454690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf2c4-435c-4950-adc1-b2010d2447ef None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.454981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf2c4-435c-4950-adc1-b2010d2447ef None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.455264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf2c4-435c-4950-adc1-b2010d2447ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.464007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:39.464693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.465004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.465289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.466024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:39.467499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:39.476744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9d5bf2c4-435c-4950-adc1-b2010d2447ef None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:39.478384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9d5bf2c4-435c-4950-adc1-b2010d2447ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:39.507438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 508/1017] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:50:39 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:50:39.509022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:39.554281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:39.617503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 510/1018] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:39.651274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bf2c4-435c-4950-adc1-b2010d2447ef None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:39.702591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3af62-d869-4148-a948-fb37d3cc11dd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:39.756627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:39.765662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1dc28-6ad5-43f0-be95-e2c28c03cc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:39.769034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1dc28-6ad5-43f0-be95-e2c28c03cc05 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.769034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1dc28-6ad5-43f0-be95-e2c28c03cc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.769034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1dc28-6ad5-43f0-be95-e2c28c03cc05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.769034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1dc28-6ad5-43f0-be95-e2c28c03cc05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:39.771852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1dc28-6ad5-43f0-be95-e2c28c03cc05 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:39.808532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 509/1019] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:50:39.836005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bf2c4-435c-4950-adc1-b2010d2447ef None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:39.912209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fa9ac-358c-4751-9fe6-98af2cc19f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:39.913084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fa9ac-358c-4751-9fe6-98af2cc19f39 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:39.913372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fa9ac-358c-4751-9fe6-98af2cc19f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:39.913652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fa9ac-358c-4751-9fe6-98af2cc19f39 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:39.914437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fa9ac-358c-4751-9fe6-98af2cc19f39 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:39.915941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fa9ac-358c-4751-9fe6-98af2cc19f39 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:39.949715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 511/1020] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:50:39 2022] POST /identity/v3/services => generated 242 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:42.168891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b8340-696d-4a67-aa70-b272dbbc4279 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.169477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b8340-696d-4a67-aa70-b272dbbc4279 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.169477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b8340-696d-4a67-aa70-b272dbbc4279 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.171056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 510/1021] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:42.177464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c747-c7b1-42d6-a456-b920bcc744c7 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.178161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c747-c7b1-42d6-a456-b920bcc744c7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.178161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c747-c7b1-42d6-a456-b920bcc744c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.195925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-eeb1c747-c7b1-42d6-a456-b920bcc744c7 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:42.197170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-eeb1c747-c7b1-42d6-a456-b920bcc744c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:42.226463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:42.257211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3a0dd-d5e2-433e-af95-42675ebf6e2f None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.257479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3a0dd-d5e2-433e-af95-42675ebf6e2f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.257758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3a0dd-d5e2-433e-af95-42675ebf6e2f None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.259404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 511/1022] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:42.264683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bed1db-9231-41d1-9685-b30615c3717b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.265120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bed1db-9231-41d1-9685-b30615c3717b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.265310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bed1db-9231-41d1-9685-b30615c3717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.283026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-92bed1db-9231-41d1-9685-b30615c3717b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:42.284232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-92bed1db-9231-41d1-9685-b30615c3717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:42.292704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 512/1023] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:42.322351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1c747-c7b1-42d6-a456-b920bcc744c7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:42.366975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 512/1024] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:42.392168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bed1db-9231-41d1-9685-b30615c3717b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:42.459751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1c747-c7b1-42d6-a456-b920bcc744c7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:42.519307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:42.521667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.522168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.522679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.544138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:42.561850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:42.562873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:42.563702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:42.564347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bed1db-9231-41d1-9685-b30615c3717b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:42.569218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 513/1025] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jun 29 07:50:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:42.572695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:42.594648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:42.597774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:42.624448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:42.625618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.625984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.626392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.628760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:42.630495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:42.632588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 513/1026] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:42.640948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:42.714595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967450-0507-4cff-ae7a-33a9e8e0a12d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:42.765953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959680-2734-424c-b882-8bc80714af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:42.767096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959680-2734-424c-b882-8bc80714af66 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.767583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959680-2734-424c-b882-8bc80714af66 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.768075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959680-2734-424c-b882-8bc80714af66 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.769627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959680-2734-424c-b882-8bc80714af66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:42.770688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959680-2734-424c-b882-8bc80714af66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:42.773325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959680-2734-424c-b882-8bc80714af66 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:42.794178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 514/1027] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jun 29 07:50:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:42.796059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959680-2734-424c-b882-8bc80714af66 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:42.845329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7264d-3732-45c5-88ce-d23772371acb None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:42.914932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:42.916064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:42.916557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:42.917057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:42.917658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:42.918435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:42.921296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:42.955864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959680-2734-424c-b882-8bc80714af66 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:42.960814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 514/1028] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:50:42 2022] POST /identity/v3/users => generated 285 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:43.012039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dad73-5011-47a9-b59f-16ff58f6c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:43.012776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dad73-5011-47a9-b59f-16ff58f6c98f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.013096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dad73-5011-47a9-b59f-16ff58f6c98f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.013382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dad73-5011-47a9-b59f-16ff58f6c98f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.027113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dad73-5011-47a9-b59f-16ff58f6c98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:43.028820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dad73-5011-47a9-b59f-16ff58f6c98f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c24dad73-5011-47a9-b59f-16ff58f6c98f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:43.049297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:43.051043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:43.053484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:43.053484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:43.053484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:43.053484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:43.053484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 515/1029] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:43.060316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:43.151619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:43.275770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9ea0-71f1-47f4-bd2a-814e5c6977db None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:43.283328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73dc3-3a65-4d7f-aa8a-fc9f71805ef5 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.284357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73dc3-3a65-4d7f-aa8a-fc9f71805ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.284651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73dc3-3a65-4d7f-aa8a-fc9f71805ef5 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.286547 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 516/1030] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:43.293019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3fe22-955d-4087-a9c9-269f742c2432 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.293677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3fe22-955d-4087-a9c9-269f742c2432 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.294193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3fe22-955d-4087-a9c9-269f742c2432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.321135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-20b3fe22-955d-4087-a9c9-269f742c2432 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:43.322388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-20b3fe22-955d-4087-a9c9-269f742c2432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:43.350283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:43.351395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.351888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.352392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.353789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:43.354325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:43.356877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:43.379128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 515/1031] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:43.386082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:43.409584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 517/1032] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:43.434405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3fe22-955d-4087-a9c9-269f742c2432 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:43.514946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:43.528781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fc598-36df-462d-a388-5892f5ecd502 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:43.531942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:43.594222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:43.595053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.595374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.595904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.603346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3fe22-955d-4087-a9c9-269f742c2432 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:43.605161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:43.606989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:43.627077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:43.629135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:43.631343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 516/1033] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:50:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:43.638512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:43.673327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:43.674177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.674484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.674731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] PATH_INFO: `/v3/services/image` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.675475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:43.677647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:43.690369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:43.692124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:43.693277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 518/1034] 10.222.0.43 () {56 vars in 1105 bytes} [Wed Jun 29 07:50:43 2022] GET /identity/v3/services/image => generated 86 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:43.702502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:43.829022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5080123-4651-4625-b2dd-879b959647aa None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:43.878086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf1aed-1c4f-42c1-99b3-d64934e49458 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:43.888757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:43.889866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.890343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.890814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.892293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:43.892819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:43.895240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:43.925688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:43.926871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 517/1035] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:50:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:43.934045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:43.946992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:43.947878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:43.948354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:43.948808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:43.949729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] RBAC: Adding query filter params (name=image) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:43.950095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:43.951861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:43.959099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 519/1036] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:43.966665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:43.972381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:44.102803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119aa2-73db-4b3a-bb43-e4dcfc006e4f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:44.140629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c990-cae5-4ac0-ab1d-048fa4a84156 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:44.170308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:44.171346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:44.171812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:44.172272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:44.198134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:44.199683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:44.199987 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:44.200274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:44.201220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] RBAC: Adding query filter params (type=image) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:44.201685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:44.203467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:44.211521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 520/1037] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:44.220458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:44.266073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:44.267762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:44.301936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:44.305666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:44.380503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9efe7-baed-48b4-8b9a-4175bc2d4122 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:44.412193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 518/1038] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 07:50:44.413986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:44.434725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:44.435407 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:44.435727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:44.436022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:44.436828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] RBAC: Adding query filter params (interface=public, service_id=f20d2756b538464fa560ac48246ecb58, region_id=RegionOne) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:44.437180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:44.438589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:44.449874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 521/1039] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:50:44 2022] GET /identity/v3/endpoints?service_id=f20d2756b538464fa560ac48246ecb58&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:44.450837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:44.573336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580e65f-d9f0-4c4e-a31a-4b069692e592 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:44.600747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91cc15-26d6-421c-b729-b94754614bcf None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:44.603214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:44.619715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:44.620519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:44.620802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:44.621114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:44.625892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:44.633547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:44.634933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:44.655047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:44.657253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:44.658610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:44.658610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:44.658610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:44.658610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:44.658610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 519/1040] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:44.658905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:44.658905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:44.658905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:44.658905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:44.659361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:44.661446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:44.663821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:44.681208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 522/1041] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:50:44 2022] GET /identity/v3/services => generated 1957 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:50:44.858627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69f108-9bb7-4af6-a905-15e2ed1a3a4d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:44.909208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:44.910674 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:44.911420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:44.912108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:44.913124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:44.913723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:44.915849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:44.934983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 520/1042] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:44.944631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:45.089775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe848-a063-4243-a52a-499b11bfd46b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:45.133629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39263194-32c7-463f-a091-7b293416f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:45.134651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263194-32c7-463f-a091-7b293416f83b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:45.135013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263194-32c7-463f-a091-7b293416f83b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:45.135300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263194-32c7-463f-a091-7b293416f83b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:45.147289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39263194-32c7-463f-a091-7b293416f83b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:45.149026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39263194-32c7-463f-a091-7b293416f83b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-39263194-32c7-463f-a091-7b293416f83b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:45.169595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:45.171358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:45.173670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:45.179387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 523/1043] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:45.179594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:45.235681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:45.342763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4ea90-761f-4d31-8991-432cb416cdb0 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:45.343298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4ea90-761f-4d31-8991-432cb416cdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:45.343888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4ea90-761f-4d31-8991-432cb416cdb0 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:45.347185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 524/1044] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:45.353888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7173e33-b0d8-49a2-a3ed-332b807e2527 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:45.354263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7173e33-b0d8-49a2-a3ed-332b807e2527 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:45.354440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7173e33-b0d8-49a2-a3ed-332b807e2527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:45.365629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777cd48-4089-42d2-9a93-c488dffc00ee None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:45.382697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a7173e33-b0d8-49a2-a3ed-332b807e2527 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:45.384294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a7173e33-b0d8-49a2-a3ed-332b807e2527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:45.432710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:45.433783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:45.434225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:45.434635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:45.435575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:45.435987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:45.437643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:45.439221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:45.461032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 521/1045] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:45.465236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:45.527068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 525/1046] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:45.575432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7173e33-b0d8-49a2-a3ed-332b807e2527 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:45.597800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:45.613529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66404940-1144-4c71-b1a0-d9b85984400d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:45.678691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:45.679726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:45.680197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:45.680667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:45.682376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] RBAC: Adding query filter params (role.id=2126aab4c72841deabc319457a5fab5c, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a267b6f64f5f49b8b3f5837e701bc216) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:45.682989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:45.685273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:45.696356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 522/1047] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:45 2022] GET /identity/v3/role_assignments?user.id=a267b6f64f5f49b8b3f5837e701bc216&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=2126aab4c72841deabc319457a5fab5c => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:45.772198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7173e33-b0d8-49a2-a3ed-332b807e2527 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:45.842017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:45.843715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:45.845121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:45.846349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:45.872094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:45.885624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:45.886663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:45.887504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:45.888839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 526/1048] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:45.898479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:46.069664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1315ff-5cc5-4d3a-9fd5-6b47f4c4efe4 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:46.130432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:46.131548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:46.132073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:46.132526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:46.133913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:46.134452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:46.136930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:46.157906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 523/1049] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:46.159499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:46.202791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:46.302986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33e044-41d3-4ecb-b296-71ff2e582b2d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:46.357899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de21f66-378a-4803-9296-a4d9718c1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:46.359291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de21f66-378a-4803-9296-a4d9718c1290 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:46.359803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de21f66-378a-4803-9296-a4d9718c1290 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:46.360292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de21f66-378a-4803-9296-a4d9718c1290 None admin] PATH_INFO: `/v3/users/7f58d6f4615544eba3953a40799c87de` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:46.385895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de21f66-378a-4803-9296-a4d9718c1290 None admin] RBAC: Authorizing `identity:get_user(user_id=7f58d6f4615544eba3953a40799c87de)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:46.388202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de21f66-378a-4803-9296-a4d9718c1290 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:46.397352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 527/1050] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:50:46 2022] GET /identity/v3/users/7f58d6f4615544eba3953a40799c87de => generated 285 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:46.401690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:46.436088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:46.586156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda1405-3982-4ca7-9db1-9eea1e19ee63 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:46.648864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:46.649864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:46.650355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:46.650831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:46.776349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:46.779068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:46.814019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:46.843011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848511-bbc1-47ee-bff4-ea2acc3645e3 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:46.843628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848511-bbc1-47ee-bff4-ea2acc3645e3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:46.843902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848511-bbc1-47ee-bff4-ea2acc3645e3 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:46.846458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 528/1051] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:46.852760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba7aac-103e-4711-ba6e-00a70a8d6b32 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:46.853168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba7aac-103e-4711-ba6e-00a70a8d6b32 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:46.853400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba7aac-103e-4711-ba6e-00a70a8d6b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:46.874168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-afba7aac-103e-4711-ba6e-00a70a8d6b32 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:46.875394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-afba7aac-103e-4711-ba6e-00a70a8d6b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:46.943387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 524/1052] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:46 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:50:46.952223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:46.977264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 529/1053] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:47.016552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba7aac-103e-4711-ba6e-00a70a8d6b32 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:47.050142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:47.096693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edc990-1464-4508-b2f8-f3e74847636d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:47.129938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:47.153394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:47.154533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:47.154914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:47.155254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:47.156735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] RBAC: Authorizing `identity:get_project(project_id=7efedb8557364aca9993a342363f03b3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:47.158852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:47.160957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 525/1054] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:50:46 2022] GET /identity/v3/projects/7efedb8557364aca9993a342363f03b3 => generated 311 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:47.162370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba7aac-103e-4711-ba6e-00a70a8d6b32 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:47.167665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:47.212348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:47.213093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:47.213377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:47.213656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] PATH_INFO: `/v3/services/image` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:47.214325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:47.215725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:47.229176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:47.230313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:47.234122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 530/1055] 10.222.0.43 () {56 vars in 1105 bytes} [Wed Jun 29 07:50:47 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:47.236708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:47.335829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:47.342501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbab8-9818-4943-909e-bda5f8b7b971 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:47.369729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80040a21-01db-4f61-872d-5842bd373e59 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:47.405110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:47.406261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:47.406784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:47.407090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:47.408753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=7efedb8557364aca9993a342363f03b3, user.id=7f58d6f4615544eba3953a40799c87de) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:47.409355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:47.412089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:47.413902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:47.414788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:47.415278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:47.415732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:47.416626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] RBAC: Adding query filter params (name=image) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:47.417150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:47.418738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:47.423413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 526/1056] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:47 2022] GET /identity/v3/role_assignments?user.id=7f58d6f4615544eba3953a40799c87de&scope.project.id=7efedb8557364aca9993a342363f03b3&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:47.435310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 531/1057] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:47.444365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:47.528074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:47.602263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7158b-cb45-4399-862a-e22961e4a723 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:47.640607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7428d5-610e-4608-9ccb-2433a47c318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:47.641630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7428d5-610e-4608-9ccb-2433a47c318d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:47.642095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7428d5-610e-4608-9ccb-2433a47c318d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:47.642546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7428d5-610e-4608-9ccb-2433a47c318d None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:47.643459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7428d5-610e-4608-9ccb-2433a47c318d None admin] RBAC: Adding query filter params (type=image) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:47.643942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7428d5-610e-4608-9ccb-2433a47c318d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:47.645604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7428d5-610e-4608-9ccb-2433a47c318d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:47.658004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 527/1058] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:47.658851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:47.789844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a7a49-68d2-4699-9965-3300bc3900ed None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:47.844808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e451c35-fbed-4020-9409-2460efd3e2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:47.845520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451c35-fbed-4020-9409-2460efd3e2e9 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:47.845827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451c35-fbed-4020-9409-2460efd3e2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:47.846081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451c35-fbed-4020-9409-2460efd3e2e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:47.846840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e451c35-fbed-4020-9409-2460efd3e2e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:47.848257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e451c35-fbed-4020-9409-2460efd3e2e9 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:47.877019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 532/1059] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:50:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 07:50:48.171242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92c8d-e8cc-411e-bb93-9a63fb732c67 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:48.171527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92c8d-e8cc-411e-bb93-9a63fb732c67 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:48.171809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92c8d-e8cc-411e-bb93-9a63fb732c67 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:48.173619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 528/1060] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:48.184048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3a154-6027-4297-bd48-9c831457b5ab None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:48.184530 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3a154-6027-4297-bd48-9c831457b5ab None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:48.185029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3a154-6027-4297-bd48-9c831457b5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:48.193910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:48.214549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e3b3a154-6027-4297-bd48-9c831457b5ab None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:48.217237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e3b3a154-6027-4297-bd48-9c831457b5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:48.348607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 533/1061] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:48.387874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92c8d-e8cc-411e-bb93-9a63fb732c67 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:48.560529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92c8d-e8cc-411e-bb93-9a63fb732c67 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:48.577745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:48.606116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:48.607003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:48.607289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:48.607565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:48.622972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:48.634175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:48.636022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:48.638052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:48.641467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 529/1062] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jun 29 07:50:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:48.643894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3a154-6027-4297-bd48-9c831457b5ab None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:48.776671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3a154-6027-4297-bd48-9c831457b5ab None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:48.820322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:48.833059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:48.833664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:48.834044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:48.834376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:48.835425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:48.835754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:48.837378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:48.849040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 534/1063] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jun 29 07:50:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:48.856499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:48.994756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da2c90-4313-47d1-87dd-2b96a907c06b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:49.040203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:49.041186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:49.041471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:49.041785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:49.055912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:49.057624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:49.090754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:49.091761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:49.093504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:49.093504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:49.093504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:49.093504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:49.093772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 530/1064] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:49.099352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:49.234344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fa635-fba7-4fd2-879f-b0337d2d60f0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:49.280589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:49.281669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:49.282021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:49.282306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:49.283048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:49.283362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:49.285216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:49.305237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 535/1065] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:49.311428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:49.362247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:49.498512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eafdb-1e1a-458d-8652-45c79667739b None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:49.560446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7943f-e422-4b89-aeb6-8d55e164f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:49.561466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7943f-e422-4b89-aeb6-8d55e164f582 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:49.562187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7943f-e422-4b89-aeb6-8d55e164f582 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:49.562187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7943f-e422-4b89-aeb6-8d55e164f582 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:49.574225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7943f-e422-4b89-aeb6-8d55e164f582 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:49.576512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7943f-e422-4b89-aeb6-8d55e164f582 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-6fd7943f-e422-4b89-aeb6-8d55e164f582 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:49.597274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:49.598381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:49.599525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:49.600429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 531/1066] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:50:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:49.605286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:49.609634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bfc88-030c-45ec-b4cd-8ab315854046 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:49.609925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bfc88-030c-45ec-b4cd-8ab315854046 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:49.610212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bfc88-030c-45ec-b4cd-8ab315854046 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:49.611731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 532/1067] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:49.618147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaa813-66e1-41af-9681-79ea7e476138 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:49.618431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaa813-66e1-41af-9681-79ea7e476138 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:49.618710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaa813-66e1-41af-9681-79ea7e476138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:49.638305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8fcaa813-66e1-41af-9681-79ea7e476138 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:49.639515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8fcaa813-66e1-41af-9681-79ea7e476138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:49.665307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:49.720625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 533/1068] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:49.725730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74747-07b9-43cb-9df9-1a92e4f45e35 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:49.743943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaa813-66e1-41af-9681-79ea7e476138 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:49.763548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:49.777642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:49.778433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:49.778642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:49.778886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:49.780030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:49.780030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:49.781534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:49.806625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 536/1069] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:50:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:49.812275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:49.880575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaa813-66e1-41af-9681-79ea7e476138 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:49.938551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:49.939833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:49.940407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:49.940747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:49.957961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:49.971322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:49.972437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:49.974185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:49.974882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc2cee-db5d-443f-80b7-70859706aa85 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:49.981330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 534/1070] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:49.983663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:49.991282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.027988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.029238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.029714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.030198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.043870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.046383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:50.067942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:50.068913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:50.070081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:50.070081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:50.070081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:50.070081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:50.076560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 537/1071] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:50.077693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.082572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.107872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68275eef-42b2-45b4-b96f-2d491b1b4d72 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.143266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e02a61-6093-4d81-99a1-744c9972559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.143928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e02a61-6093-4d81-99a1-744c9972559d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.144207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e02a61-6093-4d81-99a1-744c9972559d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.144537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e02a61-6093-4d81-99a1-744c9972559d None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.146128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e02a61-6093-4d81-99a1-744c9972559d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:50.146507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e02a61-6093-4d81-99a1-744c9972559d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.147892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e02a61-6093-4d81-99a1-744c9972559d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.156310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 535/1072] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:50.157629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079b5af-8899-432f-bc18-d880b5564443 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.157894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079b5af-8899-432f-bc18-d880b5564443 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.158196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079b5af-8899-432f-bc18-d880b5564443 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.159698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 536/1073] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:50.164796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079b5af-8899-432f-bc18-d880b5564443 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.182823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f65cb5-16c9-4a01-a389-ba04e7c3c31e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.224094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039416e8-3660-4dd5-8e5e-2ee6c768d2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.224832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039416e8-3660-4dd5-8e5e-2ee6c768d2db None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.225231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039416e8-3660-4dd5-8e5e-2ee6c768d2db None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.225583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039416e8-3660-4dd5-8e5e-2ee6c768d2db None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.226396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039416e8-3660-4dd5-8e5e-2ee6c768d2db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:50.226768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039416e8-3660-4dd5-8e5e-2ee6c768d2db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.228365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039416e8-3660-4dd5-8e5e-2ee6c768d2db None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.241002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 538/1074] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:50.242314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4ec11-65dd-42dc-8e04-fb917c27f13b None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.242682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4ec11-65dd-42dc-8e04-fb917c27f13b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.242986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4ec11-65dd-42dc-8e04-fb917c27f13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.259769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3ab4ec11-65dd-42dc-8e04-fb917c27f13b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:50.261028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3ab4ec11-65dd-42dc-8e04-fb917c27f13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:50.277639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.278709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079b5af-8899-432f-bc18-d880b5564443 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.324066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.325413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.325881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.326340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] PATH_INFO: `/v3/users/7f58d6f4615544eba3953a40799c87de` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.328115 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] RBAC: Authorizing `identity:get_user(user_id=7f58d6f4615544eba3953a40799c87de)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.330631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.333757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 537/1075] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:50:50 2022] GET /identity/v3/users/7f58d6f4615544eba3953a40799c87de => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:50.335067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.349644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.352006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 539/1076] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:50.354496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4ec11-65dd-42dc-8e04-fb917c27f13b None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.454941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625145f5-ba59-45d0-ad18-5cfb84032b7c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.494868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.495668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.495958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.496224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.503221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.505382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.505716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4ec11-65dd-42dc-8e04-fb917c27f13b None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:50.519544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:50.520675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:50.521631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 538/1077] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:50.522128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.557818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.558684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.558968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.559241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.560485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] RBAC: Authorizing `identity:get_project(project_id=7efedb8557364aca9993a342363f03b3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.562037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.563769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 540/1078] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:50:50 2022] GET /identity/v3/projects/7efedb8557364aca9993a342363f03b3 => generated 311 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:50.564543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.584442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.608129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.652871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6cee2-e127-4999-a488-74d433b985de None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.684658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02b7d5-5eb7-4600-8f35-c32d44d5ee09 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.695261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.695906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.696188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.696468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.720996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.721669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.721951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.722228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.723621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:50.724217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.725388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.737735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 541/1079] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:50.739503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.775862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.777372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.837686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.859539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:50.861982 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92dc1d-18a5-4129-aeb0-53053e78a9d0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:50.899517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:50.900436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:50.900717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:50.901030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3/users/7f58d6f4615544eba3953a40799c87de/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:50.903160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] RBAC: Authorizing `identity:create_grant(project_id=7efedb8557364aca9993a342363f03b3, user_id=7f58d6f4615544eba3953a40799c87de, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:50.904604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:50.911810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 539/1080] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:50:50.912656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:50.919618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 542/1081] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:50:50 2022] PUT /identity/v3/projects/7efedb8557364aca9993a342363f03b3/users/7f58d6f4615544eba3953a40799c87de/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:50:50.920476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:51.076898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc930-3805-46d6-b156-59f2b68c541c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:51.080925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b779c-9d62-4f77-9351-f6542985592a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:51.124772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:51.125662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:51.126005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:51.126262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/a267b6f64f5f49b8b3f5837e701bc216/roles/2126aab4c72841deabc319457a5fab5c` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:51.128654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=a267b6f64f5f49b8b3f5837e701bc216, role_id=2126aab4c72841deabc319457a5fab5c)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:51.130759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:51.136727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f032-a948-4c10-8a02-f55788191a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:51.137651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f032-a948-4c10-8a02-f55788191a9a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:51.137955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f032-a948-4c10-8a02-f55788191a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:51.138269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f032-a948-4c10-8a02-f55788191a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:51.149872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:51.153610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 540/1082] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:50:50 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/a267b6f64f5f49b8b3f5837e701bc216/roles/2126aab4c72841deabc319457a5fab5c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:50:51.155503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f032-a948-4c10-8a02-f55788191a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:51.157046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f032-a948-4c10-8a02-f55788191a9a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-8343f032-a948-4c10-8a02-f55788191a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:51.184473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:51.185871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:51.187039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:51.187039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:51.187039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:51.187039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:51.195469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 543/1083] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:51.197631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:51.232301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:51.334508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429d33-eb46-4d75-b99e-a7450d28fff6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:51.374223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1d37e-5951-402e-9458-cb4c573d78e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:51.374971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1d37e-5951-402e-9458-cb4c573d78e2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:51.375199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1d37e-5951-402e-9458-cb4c573d78e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:51.375471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1d37e-5951-402e-9458-cb4c573d78e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:51.376209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1d37e-5951-402e-9458-cb4c573d78e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:51.376518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1d37e-5951-402e-9458-cb4c573d78e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:51.377948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1d37e-5951-402e-9458-cb4c573d78e2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:51.392602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 541/1084] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:53.065108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12f52e-6953-4588-8bc7-1591d85a9fcb None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.065911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12f52e-6953-4588-8bc7-1591d85a9fcb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.066064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12f52e-6953-4588-8bc7-1591d85a9fcb None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.068076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 544/1085] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:53.074004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2492e-2bce-4b1d-bca0-96c5408627b9 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.074742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2492e-2bce-4b1d-bca0-96c5408627b9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.075035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2492e-2bce-4b1d-bca0-96c5408627b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.093298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4ff2492e-2bce-4b1d-bca0-96c5408627b9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:53.094475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4ff2492e-2bce-4b1d-bca0-96c5408627b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:53.164371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 542/1086] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:53.188919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12f52e-6953-4588-8bc7-1591d85a9fcb None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:53.304844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12f52e-6953-4588-8bc7-1591d85a9fcb None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:53.340215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f2ffd-28ff-4bba-acec-4828175d70a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:53.341210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f2ffd-28ff-4bba-acec-4828175d70a4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.341584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f2ffd-28ff-4bba-acec-4828175d70a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.341940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f2ffd-28ff-4bba-acec-4828175d70a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.353008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f2ffd-28ff-4bba-acec-4828175d70a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-748f2ffd-28ff-4bba-acec-4828175d70a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:53.360635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:53.362280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:53.364587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 545/1087] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:50:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:50:53.370098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2492e-2bce-4b1d-bca0-96c5408627b9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:53.388450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:53.459873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf9051-27b5-4b81-9a6c-94f769f02be4 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.460313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf9051-27b5-4b81-9a6c-94f769f02be4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.460583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf9051-27b5-4b81-9a6c-94f769f02be4 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.462444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 546/1088] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:53.468052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b23b62-1f07-42a0-b50f-be5290ad8ee5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.468338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b23b62-1f07-42a0-b50f-be5290ad8ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.468619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b23b62-1f07-42a0-b50f-be5290ad8ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.478359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2492e-2bce-4b1d-bca0-96c5408627b9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:53.486791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-99b23b62-1f07-42a0-b50f-be5290ad8ee5 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:53.487983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-99b23b62-1f07-42a0-b50f-be5290ad8ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:53.517028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:53.517717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.518006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.518284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.519041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:53.519348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:53.521441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:53.530172 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 543/1089] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:50:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:53.537774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:53.540093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:53.563708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 547/1090] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:53.590633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b23b62-1f07-42a0-b50f-be5290ad8ee5 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:53.627650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:53.650876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b60c0-e91a-4113-8085-bbeca8c0cd20 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:53.701594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:53.702491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.702776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.703053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] PATH_INFO: `/v3/users/7f58d6f4615544eba3953a40799c87de` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.705590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] RBAC: Authorizing `identity:get_user(user_id=7f58d6f4615544eba3953a40799c87de)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:53.708153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:53.711010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b23b62-1f07-42a0-b50f-be5290ad8ee5 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:53.712603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 544/1091] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:50:53 2022] GET /identity/v3/users/7f58d6f4615544eba3953a40799c87de => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:53.720194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:53.749552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:53.750623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.751005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.751380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.763230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:50:53.771924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:53.773158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:53.774311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 548/1092] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jun 29 07:50:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:53.779926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:53.826910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:53.857451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f2ab-c5a3-481c-91fa-e54046ff7361 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:53.892185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643b0a4-1bf0-4be1-b344-45d8439c2daa None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:53.896945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:53.897725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.898067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.898407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.915608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:53.933454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:53.934278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:53.934632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:53.934847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:53.935612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:53.935930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:53.937347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:53.946739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 549/1093] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jun 29 07:50:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:53.955450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:53.981235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:53.982714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:54.093646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e19a12-6afd-432e-b366-d382b31a59ec None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:54.095336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:54.095868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 545/1094] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:50:54.101928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:54.133410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:54.134125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:54.134569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:54.134690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:54.145849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:54.147326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:54.161066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:54.166406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:54.167384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:54.168326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:54.168326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:54.168326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:54.168326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:54.168326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 550/1095] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:54.173750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:54.229918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8635a2-4691-4ba7-a750-e785b5ed4b70 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:54.271485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:54.272371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:54.272656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:54.272938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:54.274231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] RBAC: Authorizing `identity:get_project(project_id=7efedb8557364aca9993a342363f03b3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:54.275702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:54.277515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 546/1096] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:50:54 2022] GET /identity/v3/projects/7efedb8557364aca9993a342363f03b3 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:54.286211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:54.299123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50ee26-f92e-44ea-ba44-1d3987096d91 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:54.338464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:54.339294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:54.339676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:54.340049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:54.340920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:54.341384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:54.343202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:54.357551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 551/1097] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:54.363232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:54.382531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:54.419416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da77a0c-34cf-4b9f-83fd-50d617946224 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:54.438506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:54.462017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:54.462689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:54.462974 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:54.463251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:54.464100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=7efedb8557364aca9993a342363f03b3, user.id=7f58d6f4615544eba3953a40799c87de) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:54.464418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:54.465953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:54.472307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 547/1098] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:54 2022] GET /identity/v3/role_assignments?user.id=7f58d6f4615544eba3953a40799c87de&scope.project.id=7efedb8557364aca9993a342363f03b3&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:54.492018 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66fca9-c69d-4ccc-b086-3d75432a362a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:54.532565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:54.533437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:54.533724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:54.534009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:54.543959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:54.545518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:50:54.561995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:54.563089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:50:54.564268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 552/1099] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jun 29 07:50:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:54.569763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:54.733540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905cad0-f3a9-4d26-8f92-4c1b0ae237cd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:54.777476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:54.778180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:54.778464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:54.779321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:54.779691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:54.780003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:54.781637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:54.804403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 548/1100] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jun 29 07:50:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:54.810406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:54.906226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:54.939412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6006fd6-6d36-4f91-935f-46fbbd8827cc None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:54.981201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:54.981868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:54.982159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:54.982441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:55.065225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:55.067258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:55.156326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:55.187602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 553/1101] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:50:55.193756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:55.234175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:55.312879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e10a2-d55f-4bb0-a9e4-29cf700c0766 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:55.347832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:55.348733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:55.349046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:55.349326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:55.360195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:55.361688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:50:55.380795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:55.382685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:55.384424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:50:55.384424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:50:55.384424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:50:55.384424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:55.384424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 549/1102] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:55.388568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:55.503107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b392f-6700-48aa-a593-36ce8b29af6d None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:55.543156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:55.543824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:55.544113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:55.544393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:55.545155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:55.545466 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:55.546837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:55.560338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 554/1103] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:55.566202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:55.659316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:55.673318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcebda-38ba-480a-b818-da8a6996d2b9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:55.715667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:55.716553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:55.716891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:55.717258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:55.726289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:55.727763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:50:55.750238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:50:55.751361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:50:55.752474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 550/1104] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:50:55.760775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:55.805470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:55.918733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c42cf-03c4-420c-b8a5-f2bff43884c2 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:55.978555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070269a1-410d-4153-ac67-33547cf88ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:55.979521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070269a1-410d-4153-ac67-33547cf88ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:55.979851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070269a1-410d-4153-ac67-33547cf88ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:55.980129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070269a1-410d-4153-ac67-33547cf88ef4 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:55.981467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070269a1-410d-4153-ac67-33547cf88ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:55.982031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070269a1-410d-4153-ac67-33547cf88ef4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:55.984755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070269a1-410d-4153-ac67-33547cf88ef4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:56.010773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 555/1105] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:56.012231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:56.137128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9da6c-3c90-477f-b3c7-70546b457cea None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:56.179120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7cf2f-6c13-4cd4-97b2-f7bec7f8b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:56.179701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cf2f-6c13-4cd4-97b2-f7bec7f8b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:56.179989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cf2f-6c13-4cd4-97b2-f7bec7f8b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:56.180277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cf2f-6c13-4cd4-97b2-f7bec7f8b4d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:56.181194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cf2f-6c13-4cd4-97b2-f7bec7f8b4d2 None admin] RBAC: Adding query filter params (role.id=2126aab4c72841deabc319457a5fab5c, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=a267b6f64f5f49b8b3f5837e701bc216) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:50:56.181513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cf2f-6c13-4cd4-97b2-f7bec7f8b4d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:56.183020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cf2f-6c13-4cd4-97b2-f7bec7f8b4d2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:56.190283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 551/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:50:56 2022] GET /identity/v3/role_assignments?user.id=a267b6f64f5f49b8b3f5837e701bc216&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=2126aab4c72841deabc319457a5fab5c => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:56.669075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639d4a3-c0ed-4e90-8eef-3bea1c92024f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:56.669644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639d4a3-c0ed-4e90-8eef-3bea1c92024f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:56.669644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639d4a3-c0ed-4e90-8eef-3bea1c92024f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:56.671047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 556/1107] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:50:56.677853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3303e-22f7-44af-8639-f00c2cb9a711 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:56.678418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3303e-22f7-44af-8639-f00c2cb9a711 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:56.678418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3303e-22f7-44af-8639-f00c2cb9a711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:56.697406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-74b3303e-22f7-44af-8639-f00c2cb9a711 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:56.699068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-74b3303e-22f7-44af-8639-f00c2cb9a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:56.712216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:56.788841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 552/1108] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 07:50:56.818496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639d4a3-c0ed-4e90-8eef-3bea1c92024f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:56.947110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:56.957895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639d4a3-c0ed-4e90-8eef-3bea1c92024f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:56.997213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbeb87-0df2-4886-9208-733da3261be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:56.998028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbeb87-0df2-4886-9208-733da3261be8 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:56.998313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbeb87-0df2-4886-9208-733da3261be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:56.998594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbeb87-0df2-4886-9208-733da3261be8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:56.999829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbeb87-0df2-4886-9208-733da3261be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:57.001338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbeb87-0df2-4886-9208-733da3261be8 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:57.003027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 557/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:50:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:57.010421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3303e-22f7-44af-8639-f00c2cb9a711 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:57.138884 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3303e-22f7-44af-8639-f00c2cb9a711 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:57.192584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e97b9-b75d-4e4e-ac3b-b1d9d24113f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:57.193327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e97b9-b75d-4e4e-ac3b-b1d9d24113f4 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:57.193615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e97b9-b75d-4e4e-ac3b-b1d9d24113f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:57.193895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e97b9-b75d-4e4e-ac3b-b1d9d24113f4 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:57.194637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e97b9-b75d-4e4e-ac3b-b1d9d24113f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:57.196107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e97b9-b75d-4e4e-ac3b-b1d9d24113f4 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:57.211971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:57.220081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 553/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:50:57 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:50:59.425168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d6ae1-7c03-4d40-add8-ea03c53a8422 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:59.426288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d6ae1-7c03-4d40-add8-ea03c53a8422 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:59.426288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d6ae1-7c03-4d40-add8-ea03c53a8422 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:59.427828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 558/1111] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:50:59.433643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a92e-fdf5-4aa5-a806-c3201a3e38aa None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:59.434143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a92e-fdf5-4aa5-a806-c3201a3e38aa None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:59.434258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a92e-fdf5-4aa5-a806-c3201a3e38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:59.451921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-06b0a92e-fdf5-4aa5-a806-c3201a3e38aa None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:50:59.453188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-06b0a92e-fdf5-4aa5-a806-c3201a3e38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:50:59.531260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 554/1112] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:50:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:50:59.556503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d6ae1-7c03-4d40-add8-ea03c53a8422 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:59.667136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d6ae1-7c03-4d40-add8-ea03c53a8422 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:59.705474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:59.706275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:59.706564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:59.707002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] PATH_INFO: `/v3/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:59.708769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] RBAC: Authorizing `identity:get_role(role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:59.710219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:59.711656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 559/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:50:59 2022] GET /identity/v3/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:50:59.717761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a92e-fdf5-4aa5-a806-c3201a3e38aa None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:59.821620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a92e-fdf5-4aa5-a806-c3201a3e38aa None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:50:59.849525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:59.858474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:50:59.859361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:50:59.859646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:50:59.859923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] PATH_INFO: `/v3/users/21e94c55ead84c56b2b92e5bfbb08f5d` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:50:59.875289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] RBAC: Authorizing `identity:get_user(user_id=21e94c55ead84c56b2b92e5bfbb08f5d)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:50:59.876746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:50:59.878541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 555/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:50:59 2022] GET /identity/v3/users/21e94c55ead84c56b2b92e5bfbb08f5d => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:50:59.885593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:50:59.916125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:50:59.989891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ce532-702a-4f8c-9d80-239eab0b6342 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:00.024793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:00.025479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:00.025764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:00.026042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:00.096204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:00.097759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:00.128457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:00.203405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 560/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:50:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:51:00.209682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:00.322383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493904-4e55-4998-b058-cd0ddee467bf None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:00.357914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d8cde-6dd5-4a2b-afbd-9b45fb109b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:00.358855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d8cde-6dd5-4a2b-afbd-9b45fb109b21 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:00.359191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d8cde-6dd5-4a2b-afbd-9b45fb109b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:00.359516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d8cde-6dd5-4a2b-afbd-9b45fb109b21 None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:00.360768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d8cde-6dd5-4a2b-afbd-9b45fb109b21 None admin] RBAC: Authorizing `identity:get_project(project_id=7efedb8557364aca9993a342363f03b3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:00.362337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d8cde-6dd5-4a2b-afbd-9b45fb109b21 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:00.364957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 556/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:00 2022] GET /identity/v3/projects/7efedb8557364aca9993a342363f03b3 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:00.371296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:00.478650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38319c-58c5-4429-8666-0d7bbd2afd5e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:00.510303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:00.516518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb77e2-44b3-4120-a679-47a2a8bd525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:00.517214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb77e2-44b3-4120-a679-47a2a8bd525f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:00.517502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb77e2-44b3-4120-a679-47a2a8bd525f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:00.517776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb77e2-44b3-4120-a679-47a2a8bd525f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:00.518620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb77e2-44b3-4120-a679-47a2a8bd525f None admin] RBAC: Adding query filter params (role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=7efedb8557364aca9993a342363f03b3, user.id=21e94c55ead84c56b2b92e5bfbb08f5d) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:00.518933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb77e2-44b3-4120-a679-47a2a8bd525f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:00.520446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb77e2-44b3-4120-a679-47a2a8bd525f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:00.526335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 561/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:51:00 2022] GET /identity/v3/role_assignments?user.id=21e94c55ead84c56b2b92e5bfbb08f5d&scope.project.id=7efedb8557364aca9993a342363f03b3&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:02.580405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9125-2734-47af-b503-1d2f2a486888 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:02.580956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9125-2734-47af-b503-1d2f2a486888 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:02.580956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9125-2734-47af-b503-1d2f2a486888 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:02.582901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 557/1118] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:02.588432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ae1fd-0129-4733-a3d2-41f381d7c187 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:02.588882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ae1fd-0129-4733-a3d2-41f381d7c187 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:02.588998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ae1fd-0129-4733-a3d2-41f381d7c187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:02.607589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7a4ae1fd-0129-4733-a3d2-41f381d7c187 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:02.608795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7a4ae1fd-0129-4733-a3d2-41f381d7c187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:02.680124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 562/1119] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:02.704235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9125-2734-47af-b503-1d2f2a486888 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:02.765384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:02.814055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9125-2734-47af-b503-1d2f2a486888 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:02.850321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:02.851193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:02.851478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:02.851757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] PATH_INFO: `/v3/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:02.853294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] RBAC: Authorizing `identity:get_role(role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:02.854675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:02.856073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 558/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:02 2022] GET /identity/v3/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:02.862033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ae1fd-0129-4733-a3d2-41f381d7c187 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:02.972057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ae1fd-0129-4733-a3d2-41f381d7c187 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:02.998612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:03.010641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:03.011529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:03.011813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:03.012093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] PATH_INFO: `/v3/users/21e94c55ead84c56b2b92e5bfbb08f5d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:03.013350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] RBAC: Authorizing `identity:get_user(user_id=21e94c55ead84c56b2b92e5bfbb08f5d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:03.014788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:03.016521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 563/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:02 2022] GET /identity/v3/users/21e94c55ead84c56b2b92e5bfbb08f5d => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:03.024134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:03.130699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788ed25-f5bb-4a70-b1db-6f73fdc05317 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:03.156358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:03.168573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c7899-47e9-4253-a14e-ad67252d1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:03.169470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7899-47e9-4253-a14e-ad67252d1f69 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:03.170049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7899-47e9-4253-a14e-ad67252d1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:03.170049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7899-47e9-4253-a14e-ad67252d1f69 None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:03.171210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c7899-47e9-4253-a14e-ad67252d1f69 None admin] RBAC: Authorizing `identity:get_project(project_id=7efedb8557364aca9993a342363f03b3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:03.172924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c7899-47e9-4253-a14e-ad67252d1f69 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:03.174689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 559/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:03 2022] GET /identity/v3/projects/7efedb8557364aca9993a342363f03b3 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:03.183292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:03.291715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdbf8d-00d0-4f8d-a9be-282f5a8dba32 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:03.327954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a53431-26fd-4a85-aea7-5a1006caf697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:03.328848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a53431-26fd-4a85-aea7-5a1006caf697 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:03.329158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a53431-26fd-4a85-aea7-5a1006caf697 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:03.329429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a53431-26fd-4a85-aea7-5a1006caf697 None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3/users/21e94c55ead84c56b2b92e5bfbb08f5d/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:03.331662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a53431-26fd-4a85-aea7-5a1006caf697 None admin] RBAC: Authorizing `identity:create_grant(project_id=7efedb8557364aca9993a342363f03b3, user_id=21e94c55ead84c56b2b92e5bfbb08f5d, role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:03.333806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a53431-26fd-4a85-aea7-5a1006caf697 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:03.349564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 564/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:51:03 2022] PUT /identity/v3/projects/7efedb8557364aca9993a342363f03b3/users/21e94c55ead84c56b2b92e5bfbb08f5d/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:51:05.419549 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72980a-39ce-4441-a4ac-c4948600bb1e None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:05.420081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72980a-39ce-4441-a4ac-c4948600bb1e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:05.420081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72980a-39ce-4441-a4ac-c4948600bb1e None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:05.421502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 560/1124] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:05.427441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9686a6-64a5-49bb-9dac-76b85dc83f73 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:05.427865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9686a6-64a5-49bb-9dac-76b85dc83f73 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:05.427988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9686a6-64a5-49bb-9dac-76b85dc83f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:05.444384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6f9686a6-64a5-49bb-9dac-76b85dc83f73 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:05.445587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6f9686a6-64a5-49bb-9dac-76b85dc83f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:05.492443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:05.514048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 565/1125] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:05.538502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72980a-39ce-4441-a4ac-c4948600bb1e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:05.642794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72980a-39ce-4441-a4ac-c4948600bb1e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:05.678477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:05.679269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:05.679558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:05.679841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] PATH_INFO: `/v3/roles/e07c5dadb62f42c2a351a762e8a697d9` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:05.681338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] RBAC: Authorizing `identity:get_role(role_id=e07c5dadb62f42c2a351a762e8a697d9)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:05.682722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:05.684106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 561/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:05 2022] GET /identity/v3/roles/e07c5dadb62f42c2a351a762e8a697d9 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:05.689940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9686a6-64a5-49bb-9dac-76b85dc83f73 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:05.788430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9686a6-64a5-49bb-9dac-76b85dc83f73 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:05.822122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:05.822980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:05.823262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:05.823535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] PATH_INFO: `/v3/users/21e94c55ead84c56b2b92e5bfbb08f5d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:05.824706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] RBAC: Authorizing `identity:get_user(user_id=21e94c55ead84c56b2b92e5bfbb08f5d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:05.826164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:05.827783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 566/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:05 2022] GET /identity/v3/users/21e94c55ead84c56b2b92e5bfbb08f5d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:05.833720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:05.892707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:05.933996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7780b-d5c6-406a-968c-3fa17b8bbbb4 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:05.967708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:05.968358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:05.968645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:05.968925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:06.035636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:06.037112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:06.092964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:06.135832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 562/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:51:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:51:06.141752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:06.194274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:06.252872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6782e-69e4-4056-8cb5-0dcb79bb3c2c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:06.288657 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cea8d0-b34b-4775-8e25-2035be045570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:06.289560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cea8d0-b34b-4775-8e25-2035be045570 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:06.289849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cea8d0-b34b-4775-8e25-2035be045570 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:06.290123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cea8d0-b34b-4775-8e25-2035be045570 None admin] PATH_INFO: `/v3/projects/7efedb8557364aca9993a342363f03b3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:06.291347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cea8d0-b34b-4775-8e25-2035be045570 None admin] RBAC: Authorizing `identity:get_project(project_id=7efedb8557364aca9993a342363f03b3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:06.292794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cea8d0-b34b-4775-8e25-2035be045570 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:06.294548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 567/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:06 2022] GET /identity/v3/projects/7efedb8557364aca9993a342363f03b3 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:06.302214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:06.401903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfad0b9-822a-4f88-8ae0-31b0cb2fee1d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:06.436649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cce43-224c-4440-8597-99b2ffa537bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:06.437347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cce43-224c-4440-8597-99b2ffa537bf None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:06.437603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cce43-224c-4440-8597-99b2ffa537bf None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:06.437878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cce43-224c-4440-8597-99b2ffa537bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:06.438711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cce43-224c-4440-8597-99b2ffa537bf None admin] RBAC: Adding query filter params (role.id=e07c5dadb62f42c2a351a762e8a697d9, scope.project.id=7efedb8557364aca9993a342363f03b3, user.id=21e94c55ead84c56b2b92e5bfbb08f5d) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:06.439018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cce43-224c-4440-8597-99b2ffa537bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:06.440459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cce43-224c-4440-8597-99b2ffa537bf None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:06.447439 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 563/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:51:06 2022] GET /identity/v3/role_assignments?user.id=21e94c55ead84c56b2b92e5bfbb08f5d&scope.project.id=7efedb8557364aca9993a342363f03b3&role.id=e07c5dadb62f42c2a351a762e8a697d9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:51:08.624563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82b11b-e2c6-41a4-aeb9-5281665fc8a8 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:08.625134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82b11b-e2c6-41a4-aeb9-5281665fc8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:08.625134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82b11b-e2c6-41a4-aeb9-5281665fc8a8 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:08.626542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 568/1131] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:51:08.632122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35c4b-404b-4c1d-b7dc-c80d641a1279 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:08.632586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35c4b-404b-4c1d-b7dc-c80d641a1279 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:08.632676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35c4b-404b-4c1d-b7dc-c80d641a1279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:08.649495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7dd35c4b-404b-4c1d-b7dc-c80d641a1279 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:08.650689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7dd35c4b-404b-4c1d-b7dc-c80d641a1279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:08.668282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:08.734193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 564/1132] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:08.758371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82b11b-e2c6-41a4-aeb9-5281665fc8a8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:08.867634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82b11b-e2c6-41a4-aeb9-5281665fc8a8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:08.885767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:08.903968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d034f8-c0de-40b9-937b-587a03bf226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:08.904766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d034f8-c0de-40b9-937b-587a03bf226d None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:08.905071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d034f8-c0de-40b9-937b-587a03bf226d None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:08.905348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d034f8-c0de-40b9-937b-587a03bf226d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:08.906542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d034f8-c0de-40b9-937b-587a03bf226d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:08.907988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d034f8-c0de-40b9-937b-587a03bf226d None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:08.910243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 569/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:51:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:08.916455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35c4b-404b-4c1d-b7dc-c80d641a1279 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:09.022617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35c4b-404b-4c1d-b7dc-c80d641a1279 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:09.058937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b342a5b-85a3-4d75-92fe-db71c0eeb841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:09.059609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b342a5b-85a3-4d75-92fe-db71c0eeb841 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:09.059890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b342a5b-85a3-4d75-92fe-db71c0eeb841 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:09.060159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b342a5b-85a3-4d75-92fe-db71c0eeb841 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:09.061086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b342a5b-85a3-4d75-92fe-db71c0eeb841 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:09.062525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b342a5b-85a3-4d75-92fe-db71c0eeb841 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:09.090127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 565/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:51:08 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:51:11.202685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ede78-d569-43cf-920c-bbf69d9a23e0 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:11.203214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ede78-d569-43cf-920c-bbf69d9a23e0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:11.203214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ede78-d569-43cf-920c-bbf69d9a23e0 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:11.204603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 570/1135] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:51:11.210119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819a75-d4fc-44ff-adae-ddc586c44f4a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:11.210520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819a75-d4fc-44ff-adae-ddc586c44f4a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:11.210682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819a75-d4fc-44ff-adae-ddc586c44f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:11.216822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:11.229306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7f819a75-d4fc-44ff-adae-ddc586c44f4a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:11.230489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7f819a75-d4fc-44ff-adae-ddc586c44f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:11.296461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 566/1136] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:11.323056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ede78-d569-43cf-920c-bbf69d9a23e0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:11.430023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ede78-d569-43cf-920c-bbf69d9a23e0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:11.463938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecb2dc-8736-4a5c-8a7c-753f7820ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:11.464670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecb2dc-8736-4a5c-8a7c-753f7820ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:11.464953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecb2dc-8736-4a5c-8a7c-753f7820ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:11.465255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecb2dc-8736-4a5c-8a7c-753f7820ccf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:11.466422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecb2dc-8736-4a5c-8a7c-753f7820ccf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:11.467853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecb2dc-8736-4a5c-8a7c-753f7820ccf0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:11.469566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 571/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:51:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:11.476263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f819a75-d4fc-44ff-adae-ddc586c44f4a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:11.587259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f819a75-d4fc-44ff-adae-ddc586c44f4a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:11.625134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83710941-9eae-4d6d-8464-081ff8c30fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:11.625801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83710941-9eae-4d6d-8464-081ff8c30fc4 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:11.626086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83710941-9eae-4d6d-8464-081ff8c30fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:11.626366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83710941-9eae-4d6d-8464-081ff8c30fc4 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:11.627071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83710941-9eae-4d6d-8464-081ff8c30fc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:11.629044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83710941-9eae-4d6d-8464-081ff8c30fc4 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:11.635524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:11.648715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 567/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:51:11 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:51:13.749903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60fffe-4f5f-44f0-a97c-1a90f7d182a0 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:13.750438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60fffe-4f5f-44f0-a97c-1a90f7d182a0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:13.750438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60fffe-4f5f-44f0-a97c-1a90f7d182a0 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:13.751758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 572/1139] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:13.757558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd07409-a106-4329-af7d-2524f607efa4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:13.757990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd07409-a106-4329-af7d-2524f607efa4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:13.758105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd07409-a106-4329-af7d-2524f607efa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:13.774680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fdd07409-a106-4329-af7d-2524f607efa4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:13.775868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fdd07409-a106-4329-af7d-2524f607efa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:13.842041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 568/1140] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:13.865242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60fffe-4f5f-44f0-a97c-1a90f7d182a0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:13.920950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:13.966521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60fffe-4f5f-44f0-a97c-1a90f7d182a0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:13.999414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:14.000259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:14.000489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:14.000769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:14.011275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:51:14.018950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:51:14.020101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:51:14.021044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 573/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:51:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:51:14.026894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd07409-a106-4329-af7d-2524f607efa4 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:14.124074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd07409-a106-4329-af7d-2524f607efa4 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:14.145061 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:14.159619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:14.160510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:14.160792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:14.161104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:14.161853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:14.162157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:14.163524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:14.171567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 569/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:51:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:14.178742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:14.278509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fa1ef-985d-4bd8-a632-58b67d315694 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:14.291599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:14.312457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:14.313355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:14.313636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:14.313909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] PATH_INFO: `/v3/users/f104687dec684160a087ac9ac8360c6a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:14.327221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=f104687dec684160a087ac9ac8360c6a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:14.328648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:14.330323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 574/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:14 2022] GET /identity/v3/users/f104687dec684160a087ac9ac8360c6a => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:14.336266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:14.437593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b929f2-fd7e-4163-83b3-486300592a7f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:14.473615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:14.474260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:14.474543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:14.474817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:14.523122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:14.542089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:14.543462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:14.641161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 570/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:51:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:51:14.647205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:14.747072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e052f-5ace-4932-894c-6ae07e198bf4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:14.782384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a72c8-327f-4f23-8c60-1b3c97cef258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:14.783239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a72c8-327f-4f23-8c60-1b3c97cef258 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:14.783518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a72c8-327f-4f23-8c60-1b3c97cef258 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:14.783786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a72c8-327f-4f23-8c60-1b3c97cef258 None admin] PATH_INFO: `/v3/projects/a13ec29ac65a4de284bebac329d59aa2` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:14.785547 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a72c8-327f-4f23-8c60-1b3c97cef258 None admin] RBAC: Authorizing `identity:get_project(project_id=a13ec29ac65a4de284bebac329d59aa2)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:14.786985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a72c8-327f-4f23-8c60-1b3c97cef258 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:14.788671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 575/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:14 2022] GET /identity/v3/projects/a13ec29ac65a4de284bebac329d59aa2 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:14.794765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:14.874506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:14.892556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbafc3-6098-4e23-a02a-2095308756e2 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:14.926828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3641a8d-d81f-4963-a63f-7a19734de32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:14.927475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641a8d-d81f-4963-a63f-7a19734de32f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:14.927758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641a8d-d81f-4963-a63f-7a19734de32f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:14.928031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641a8d-d81f-4963-a63f-7a19734de32f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:14.928851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3641a8d-d81f-4963-a63f-7a19734de32f None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=a13ec29ac65a4de284bebac329d59aa2, user.id=f104687dec684160a087ac9ac8360c6a) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:14.929187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3641a8d-d81f-4963-a63f-7a19734de32f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:14.930613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3641a8d-d81f-4963-a63f-7a19734de32f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:14.936021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 571/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:51:14 2022] GET /identity/v3/role_assignments?user.id=f104687dec684160a087ac9ac8360c6a&scope.project.id=a13ec29ac65a4de284bebac329d59aa2&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:16.978638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99974a-d456-45eb-afe8-e568777985d7 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:16.979195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99974a-d456-45eb-afe8-e568777985d7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:16.979195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99974a-d456-45eb-afe8-e568777985d7 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:16.980561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 576/1147] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:16.986155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713402a-562e-4dc4-b8c6-b424ae75f227 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:16.986559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713402a-562e-4dc4-b8c6-b424ae75f227 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:16.986703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713402a-562e-4dc4-b8c6-b424ae75f227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:17.002827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b713402a-562e-4dc4-b8c6-b424ae75f227 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:17.004659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b713402a-562e-4dc4-b8c6-b424ae75f227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:17.069230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 572/1148] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:17.091777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99974a-d456-45eb-afe8-e568777985d7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:17.134686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:17.192651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99974a-d456-45eb-afe8-e568777985d7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:17.226459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:17.227243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:17.227532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:17.227807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:17.239137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:51:17.246679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:51:17.247770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:51:17.248645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 577/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:51:17.254280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713402a-562e-4dc4-b8c6-b424ae75f227 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:17.332223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:17.358256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713402a-562e-4dc4-b8c6-b424ae75f227 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:17.391673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:17.392330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:17.392616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:17.392899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:17.394193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:17.394509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:17.395868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:17.403754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 573/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:51:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:17.410151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:17.506739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:17.509762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bf296-c6d4-407e-81b3-10995c8516b3 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:17.542666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:17.543526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:17.543808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:17.544085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] PATH_INFO: `/v3/users/f104687dec684160a087ac9ac8360c6a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:17.545275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] RBAC: Authorizing `identity:get_user(user_id=f104687dec684160a087ac9ac8360c6a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:17.546696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:17.548570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 578/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:17 2022] GET /identity/v3/users/f104687dec684160a087ac9ac8360c6a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:17.555042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:17.652327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56d743-7201-4625-ba7d-005ecfcd8755 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:17.686045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a293e6-43a3-499a-99ff-4ffa49850865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:17.686902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a293e6-43a3-499a-99ff-4ffa49850865 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:17.687185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a293e6-43a3-499a-99ff-4ffa49850865 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:17.687463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a293e6-43a3-499a-99ff-4ffa49850865 None admin] PATH_INFO: `/v3/projects/a13ec29ac65a4de284bebac329d59aa2` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:17.688633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a293e6-43a3-499a-99ff-4ffa49850865 None admin] RBAC: Authorizing `identity:get_project(project_id=a13ec29ac65a4de284bebac329d59aa2)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:17.690095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a293e6-43a3-499a-99ff-4ffa49850865 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:17.692056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 574/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:17 2022] GET /identity/v3/projects/a13ec29ac65a4de284bebac329d59aa2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:17.698033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:17.798001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb037-f30d-4aec-88be-cee9575da71e None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:17.830540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a12192-d68f-4757-bae4-4488799adde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:17.831969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a12192-d68f-4757-bae4-4488799adde3 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:17.832250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a12192-d68f-4757-bae4-4488799adde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:17.832523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a12192-d68f-4757-bae4-4488799adde3 None admin] PATH_INFO: `/v3/projects/a13ec29ac65a4de284bebac329d59aa2/users/f104687dec684160a087ac9ac8360c6a/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:17.834490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a12192-d68f-4757-bae4-4488799adde3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a13ec29ac65a4de284bebac329d59aa2, user_id=f104687dec684160a087ac9ac8360c6a, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:17.835914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a12192-d68f-4757-bae4-4488799adde3 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:17.849346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 579/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:51:17 2022] PUT /identity/v3/projects/a13ec29ac65a4de284bebac329d59aa2/users/f104687dec684160a087ac9ac8360c6a/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:51:19.930915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2914a7-c776-45da-9ba8-8151ce988c8a None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:19.931472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2914a7-c776-45da-9ba8-8151ce988c8a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:19.931472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2914a7-c776-45da-9ba8-8151ce988c8a None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:19.932850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 575/1154] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:19.938065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611834d5-e077-470e-9a95-9c80b9d15272 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:19.938473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611834d5-e077-470e-9a95-9c80b9d15272 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:19.938622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611834d5-e077-470e-9a95-9c80b9d15272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:19.955536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-611834d5-e077-470e-9a95-9c80b9d15272 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:19.957411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-611834d5-e077-470e-9a95-9c80b9d15272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:19.980043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:20.024136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 580/1155] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:20.047106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2914a7-c776-45da-9ba8-8151ce988c8a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:20.057287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:20.150976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2914a7-c776-45da-9ba8-8151ce988c8a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:20.185132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:20.185885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:20.186171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:20.186452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:20.196591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:51:20.204235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:51:20.205338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:51:20.206911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 576/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:51:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:51:20.211924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611834d5-e077-470e-9a95-9c80b9d15272 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:20.310078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611834d5-e077-470e-9a95-9c80b9d15272 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:20.343171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:20.343833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:20.344115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:20.344391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:20.345164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:20.345471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:20.346883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:20.354966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 581/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:51:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:51:20.360747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:20.431549 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:20.467789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722025f-e9f6-4ddd-a061-661720a95e9d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:20.504586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:20.505493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:20.505779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:20.506056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] PATH_INFO: `/v3/users/f104687dec684160a087ac9ac8360c6a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:20.507271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] RBAC: Authorizing `identity:get_user(user_id=f104687dec684160a087ac9ac8360c6a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:20.508707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:20.510390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 577/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:20 2022] GET /identity/v3/users/f104687dec684160a087ac9ac8360c6a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:20.516110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:20.538678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:20.614561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23eb9d-80c5-4ae4-8df9-e2b46491888b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:20.645955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:20.646594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:20.646876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:20.647152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:20.711809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:20.713279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:20.734000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:20.810559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 582/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:51:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:51:20.816924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:20.917134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30d986-b7de-4fa8-bc31-2d3307f5c87f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:20.950589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8a2db-5abc-4bbd-aed1-c0713d97b1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:20.951448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a2db-5abc-4bbd-aed1-c0713d97b1cd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:20.951732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a2db-5abc-4bbd-aed1-c0713d97b1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:20.952005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a2db-5abc-4bbd-aed1-c0713d97b1cd None admin] PATH_INFO: `/v3/projects/a13ec29ac65a4de284bebac329d59aa2` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:20.953203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8a2db-5abc-4bbd-aed1-c0713d97b1cd None admin] RBAC: Authorizing `identity:get_project(project_id=a13ec29ac65a4de284bebac329d59aa2)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:20.954637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8a2db-5abc-4bbd-aed1-c0713d97b1cd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:20.957447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 578/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:20 2022] GET /identity/v3/projects/a13ec29ac65a4de284bebac329d59aa2 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:20.963546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:21.060302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e260a-3b1a-41bd-89b7-b55695093d34 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:21.079596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:21.094748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd07665-721d-41b1-8e28-46ebdc6ac2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:21.095406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd07665-721d-41b1-8e28-46ebdc6ac2ff None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:21.095694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd07665-721d-41b1-8e28-46ebdc6ac2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:21.095970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd07665-721d-41b1-8e28-46ebdc6ac2ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:21.097377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd07665-721d-41b1-8e28-46ebdc6ac2ff None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=a13ec29ac65a4de284bebac329d59aa2, user.id=f104687dec684160a087ac9ac8360c6a) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:21.097705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd07665-721d-41b1-8e28-46ebdc6ac2ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:21.099131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd07665-721d-41b1-8e28-46ebdc6ac2ff None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:21.104971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 583/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:51:20 2022] GET /identity/v3/role_assignments?user.id=f104687dec684160a087ac9ac8360c6a&scope.project.id=a13ec29ac65a4de284bebac329d59aa2&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:23.271816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb049e-dbf0-4ddd-9a81-832d9c9281ee None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:23.272340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb049e-dbf0-4ddd-9a81-832d9c9281ee None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:23.272428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb049e-dbf0-4ddd-9a81-832d9c9281ee None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:23.273780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 579/1162] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:23.279127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828ea8-316b-4fff-8709-c406e001a7dc None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:23.279565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828ea8-316b-4fff-8709-c406e001a7dc None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:23.279681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828ea8-316b-4fff-8709-c406e001a7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:23.296498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-58828ea8-316b-4fff-8709-c406e001a7dc None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:23.297959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-58828ea8-316b-4fff-8709-c406e001a7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:23.366625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 584/1163] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:23.390228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb049e-dbf0-4ddd-9a81-832d9c9281ee None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:23.393156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:23.494966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb049e-dbf0-4ddd-9a81-832d9c9281ee None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:23.528348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:23.528906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:23.529219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:23.529499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:23.595578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:23.597665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:23.599498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:23.696822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 580/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:51:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:51:23.702826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828ea8-316b-4fff-8709-c406e001a7dc None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:23.800269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828ea8-316b-4fff-8709-c406e001a7dc None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:23.815672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:23.835943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddf726-85a7-4a42-9ffc-73d8602890b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:23.836739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddf726-85a7-4a42-9ffc-73d8602890b1 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:23.837053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddf726-85a7-4a42-9ffc-73d8602890b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:23.837326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddf726-85a7-4a42-9ffc-73d8602890b1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:23.843878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddf726-85a7-4a42-9ffc-73d8602890b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:23.845332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddf726-85a7-4a42-9ffc-73d8602890b1 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-7cddf726-85a7-4a42-9ffc-73d8602890b1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:51:23.859314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:51:23.860532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:51:23.861406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 585/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:51:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:51:23.867162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:23.964801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45b2c5-385f-4863-9f0d-a72dbf14dac1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:23.966369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:23.999873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca16f37-f449-403c-804d-5d2ad0d66f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:24.000533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca16f37-f449-403c-804d-5d2ad0d66f7a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:24.000813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca16f37-f449-403c-804d-5d2ad0d66f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:24.001114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca16f37-f449-403c-804d-5d2ad0d66f7a None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:24.001829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca16f37-f449-403c-804d-5d2ad0d66f7a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:24.002133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca16f37-f449-403c-804d-5d2ad0d66f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:24.003524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca16f37-f449-403c-804d-5d2ad0d66f7a None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:24.010645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 581/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jun 29 07:51:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:26.030316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d77f-3b04-444a-89ff-815619f51278 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:26.030918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d77f-3b04-444a-89ff-815619f51278 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:26.030918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d77f-3b04-444a-89ff-815619f51278 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:26.032237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 586/1167] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:26.038357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51057d4b-a101-4f9d-ba93-3d17e1fa8e3f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:26.038804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51057d4b-a101-4f9d-ba93-3d17e1fa8e3f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:26.038915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51057d4b-a101-4f9d-ba93-3d17e1fa8e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:26.056319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-51057d4b-a101-4f9d-ba93-3d17e1fa8e3f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:26.057536 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-51057d4b-a101-4f9d-ba93-3d17e1fa8e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:26.124429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 582/1168] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:26.147986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324d77f-3b04-444a-89ff-815619f51278 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:26.250407 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324d77f-3b04-444a-89ff-815619f51278 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:26.283886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1ee25-8085-4a5a-a70c-7846e6b6dd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:26.284596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1ee25-8085-4a5a-a70c-7846e6b6dd8f None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:26.284884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1ee25-8085-4a5a-a70c-7846e6b6dd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:26.285194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1ee25-8085-4a5a-a70c-7846e6b6dd8f None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:26.285946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1ee25-8085-4a5a-a70c-7846e6b6dd8f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:26.287321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1ee25-8085-4a5a-a70c-7846e6b6dd8f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:26.303173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 587/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Wed Jun 29 07:51:26 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:51:28.383494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada027cd-b42d-4f03-ae59-454149db06df None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:28.384066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada027cd-b42d-4f03-ae59-454149db06df None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:28.384066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada027cd-b42d-4f03-ae59-454149db06df None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:28.385455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 583/1170] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:28.390753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45c787-09cb-47df-be05-6e101d9d1c5a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:28.391151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45c787-09cb-47df-be05-6e101d9d1c5a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:28.391312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45c787-09cb-47df-be05-6e101d9d1c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:28.408124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-de45c787-09cb-47df-be05-6e101d9d1c5a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:28.409928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-de45c787-09cb-47df-be05-6e101d9d1c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:28.437056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:28.479509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 588/1171] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:28.502499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada027cd-b42d-4f03-ae59-454149db06df None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:28.603838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:28.610840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada027cd-b42d-4f03-ae59-454149db06df None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:28.642650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23255dc-276d-4f8b-bc85-734e283b60a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:28.643958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23255dc-276d-4f8b-bc85-734e283b60a6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:28.644238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23255dc-276d-4f8b-bc85-734e283b60a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:28.644510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23255dc-276d-4f8b-bc85-734e283b60a6 None admin] PATH_INFO: `/v3/domains/ce57c12785b74f2e8ecd8eed357af382` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:28.654694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23255dc-276d-4f8b-bc85-734e283b60a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce57c12785b74f2e8ecd8eed357af382)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:28.656130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23255dc-276d-4f8b-bc85-734e283b60a6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:28.657799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 584/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Wed Jun 29 07:51:28 2022] GET /identity/v3/domains/ce57c12785b74f2e8ecd8eed357af382 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:28.664422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45c787-09cb-47df-be05-6e101d9d1c5a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:28.759991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45c787-09cb-47df-be05-6e101d9d1c5a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:28.793349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ba2a6-a551-4c49-8a84-7fe575324617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:28.794014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ba2a6-a551-4c49-8a84-7fe575324617 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:28.794294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ba2a6-a551-4c49-8a84-7fe575324617 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:28.794566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ba2a6-a551-4c49-8a84-7fe575324617 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:28.795282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ba2a6-a551-4c49-8a84-7fe575324617 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:28.796712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ba2a6-a551-4c49-8a84-7fe575324617 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:28.824269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 589/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:51:28 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:51:30.921310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fce56b-e930-4c23-abe6-e5983efbb79b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:30.921836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fce56b-e930-4c23-abe6-e5983efbb79b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:30.921946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fce56b-e930-4c23-abe6-e5983efbb79b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:30.923502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 585/1174] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:30.929344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972782-3e76-4d23-b4b3-9cd107890f8a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:30.929753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972782-3e76-4d23-b4b3-9cd107890f8a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:30.929902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972782-3e76-4d23-b4b3-9cd107890f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:30.947059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7c972782-3e76-4d23-b4b3-9cd107890f8a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:30.948239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7c972782-3e76-4d23-b4b3-9cd107890f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:30.959255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:31.015164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 590/1175] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:31.040018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fce56b-e930-4c23-abe6-e5983efbb79b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:31.140324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fce56b-e930-4c23-abe6-e5983efbb79b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:31.175283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0bcb5-decd-4636-a71f-785e6ccdbe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:31.176211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0bcb5-decd-4636-a71f-785e6ccdbe87 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:31.176500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0bcb5-decd-4636-a71f-785e6ccdbe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:31.176777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0bcb5-decd-4636-a71f-785e6ccdbe87 None admin] PATH_INFO: `/v3/domains/ce57c12785b74f2e8ecd8eed357af382` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:31.177983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0bcb5-decd-4636-a71f-785e6ccdbe87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce57c12785b74f2e8ecd8eed357af382)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:31.179419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0bcb5-decd-4636-a71f-785e6ccdbe87 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:31.181062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 586/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Wed Jun 29 07:51:31 2022] GET /identity/v3/domains/ce57c12785b74f2e8ecd8eed357af382 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:31.186782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972782-3e76-4d23-b4b3-9cd107890f8a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:31.291036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972782-3e76-4d23-b4b3-9cd107890f8a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:31.324335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604947de-14da-4728-b035-720e437263d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:31.325017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604947de-14da-4728-b035-720e437263d6 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:31.325306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604947de-14da-4728-b035-720e437263d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:31.325580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604947de-14da-4728-b035-720e437263d6 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:31.326295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604947de-14da-4728-b035-720e437263d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:31.327717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604947de-14da-4728-b035-720e437263d6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:31.348528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 591/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:51:31 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:51:33.420621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5674d-2131-450e-8355-7fe644e03453 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:33.421222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5674d-2131-450e-8355-7fe644e03453 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:33.421222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5674d-2131-450e-8355-7fe644e03453 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:33.422839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 587/1178] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:33.428946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51718e6c-528b-4ae4-b24f-3aab452160f9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:33.429417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51718e6c-528b-4ae4-b24f-3aab452160f9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:33.429539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51718e6c-528b-4ae4-b24f-3aab452160f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:33.434515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:33.448601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-51718e6c-528b-4ae4-b24f-3aab452160f9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:33.449841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-51718e6c-528b-4ae4-b24f-3aab452160f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:33.518797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 592/1179] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:33.542381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5674d-2131-450e-8355-7fe644e03453 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:33.569145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:33.644944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5674d-2131-450e-8355-7fe644e03453 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:33.678556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:33.679259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:33.679539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:33.679812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:33.689817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:51:33.697460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:51:33.698624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:51:33.699963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 588/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:51:33.704839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51718e6c-528b-4ae4-b24f-3aab452160f9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:33.801940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51718e6c-528b-4ae4-b24f-3aab452160f9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:33.830633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:33.835884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:33.836550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:33.836832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:33.837138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:33.837889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:33.838195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:33.839569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:33.847809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 593/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:33.853655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:33.944369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:33.962002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63dc85-b225-4d5e-8df8-6a6909bff70c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:34.000095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:34.001013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:34.001303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:34.001578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] PATH_INFO: `/v3/users/d83657bfd4e040e7878cf2a1b0c4481d` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:34.016840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] RBAC: Authorizing `identity:get_user(user_id=d83657bfd4e040e7878cf2a1b0c4481d)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:34.019096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:34.020925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 589/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:33 2022] GET /identity/v3/users/d83657bfd4e040e7878cf2a1b0c4481d => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:34.027143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:34.136238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22145a-348b-46de-897e-2334192d2474 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:34.171338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:34.172015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:34.172304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:34.172837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:34.232481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:34.240451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:34.241964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:34.340425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 594/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:51:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:51:34.347176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:34.446531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa04af7-f114-4bd1-99a3-87a8923ee61f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:34.480309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5e60f-2c1c-4d5b-b242-897fb76fc027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:34.481218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e60f-2c1c-4d5b-b242-897fb76fc027 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:34.481508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e60f-2c1c-4d5b-b242-897fb76fc027 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:34.481797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e60f-2c1c-4d5b-b242-897fb76fc027 None admin] PATH_INFO: `/v3/projects/b3dafdff3aa64e819a178591032ded55` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:34.483052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e60f-2c1c-4d5b-b242-897fb76fc027 None admin] RBAC: Authorizing `identity:get_project(project_id=b3dafdff3aa64e819a178591032ded55)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:34.484451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e60f-2c1c-4d5b-b242-897fb76fc027 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:34.486399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 590/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:34 2022] GET /identity/v3/projects/b3dafdff3aa64e819a178591032ded55 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:34.493336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:34.585775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:34.592792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6400-e669-4ef8-9c72-a7eda9a6d5c4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:34.627128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193dd9b-94a6-4905-9215-667b846f068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:34.628813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193dd9b-94a6-4905-9215-667b846f068e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:34.629128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193dd9b-94a6-4905-9215-667b846f068e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:34.629408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193dd9b-94a6-4905-9215-667b846f068e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:34.630238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193dd9b-94a6-4905-9215-667b846f068e None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=b3dafdff3aa64e819a178591032ded55, user.id=d83657bfd4e040e7878cf2a1b0c4481d) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:34.630553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193dd9b-94a6-4905-9215-667b846f068e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:34.631980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193dd9b-94a6-4905-9215-667b846f068e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:34.637434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 595/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:51:34 2022] GET /identity/v3/role_assignments?user.id=d83657bfd4e040e7878cf2a1b0c4481d&scope.project.id=b3dafdff3aa64e819a178591032ded55&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:51:36.664867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30235bd-0c2f-4636-beec-3b098d8abf59 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:36.665452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30235bd-0c2f-4636-beec-3b098d8abf59 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:36.665452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30235bd-0c2f-4636-beec-3b098d8abf59 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:36.666842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 591/1186] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:36.672362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d253f-a143-4d6c-8f5b-c5cf15bd4897 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:36.673051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d253f-a143-4d6c-8f5b-c5cf15bd4897 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:36.673203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d253f-a143-4d6c-8f5b-c5cf15bd4897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:36.692495 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c32d253f-a143-4d6c-8f5b-c5cf15bd4897 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:36.693772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c32d253f-a143-4d6c-8f5b-c5cf15bd4897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:36.771267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 596/1187] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:36.794721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30235bd-0c2f-4636-beec-3b098d8abf59 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:36.809096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:36.900069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30235bd-0c2f-4636-beec-3b098d8abf59 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:36.933584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:36.934374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:36.934658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:36.934931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:36.945434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:51:36.953019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:51:36.954134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:51:36.955020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 592/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:51:36.961022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d253f-a143-4d6c-8f5b-c5cf15bd4897 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:37.047278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:37.060327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d253f-a143-4d6c-8f5b-c5cf15bd4897 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:37.093810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:37.094487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:37.094772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:37.095050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:37.095805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:37.096703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:37.098071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:37.106275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 597/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:37.113165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:37.193935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:37.227493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f010c3b-bff4-4dd2-8983-d1afe56ba108 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:37.260923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:37.261820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:37.262101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:37.262383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] PATH_INFO: `/v3/users/d83657bfd4e040e7878cf2a1b0c4481d` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:37.263573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] RBAC: Authorizing `identity:get_user(user_id=d83657bfd4e040e7878cf2a1b0c4481d)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:37.265033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:37.266665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 593/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:37 2022] GET /identity/v3/users/d83657bfd4e040e7878cf2a1b0c4481d => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:37.273372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:37.374414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a81c0-fd11-4e24-98f6-a6e012acb264 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:37.407712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cf98f-4445-4209-bc10-33246536baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:37.409158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf98f-4445-4209-bc10-33246536baba None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:37.409440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf98f-4445-4209-bc10-33246536baba None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:37.409896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf98f-4445-4209-bc10-33246536baba None admin] PATH_INFO: `/v3/projects/b3dafdff3aa64e819a178591032ded55` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:37.411070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cf98f-4445-4209-bc10-33246536baba None admin] RBAC: Authorizing `identity:get_project(project_id=b3dafdff3aa64e819a178591032ded55)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:37.412508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cf98f-4445-4209-bc10-33246536baba None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:37.414231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 598/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:37 2022] GET /identity/v3/projects/b3dafdff3aa64e819a178591032ded55 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:37.420181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:37.529155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f94ad-0581-4376-b5c1-3da7fd80ff92 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:37.563473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3120e-404c-446c-8cd9-69f3b7f9b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:37.564360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3120e-404c-446c-8cd9-69f3b7f9b1a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:37.564686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3120e-404c-446c-8cd9-69f3b7f9b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:37.565313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3120e-404c-446c-8cd9-69f3b7f9b1a2 None admin] PATH_INFO: `/v3/projects/b3dafdff3aa64e819a178591032ded55/users/d83657bfd4e040e7878cf2a1b0c4481d/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:37.567420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3120e-404c-446c-8cd9-69f3b7f9b1a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3dafdff3aa64e819a178591032ded55, user_id=d83657bfd4e040e7878cf2a1b0c4481d, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:37.568854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3120e-404c-446c-8cd9-69f3b7f9b1a2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:37.576805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:37.583637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 594/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:51:37 2022] PUT /identity/v3/projects/b3dafdff3aa64e819a178591032ded55/users/d83657bfd4e040e7878cf2a1b0c4481d/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:51:39.597813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d028dd-bffc-4581-b30d-e9c202c318b7 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:39.598361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d028dd-bffc-4581-b30d-e9c202c318b7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:39.598361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d028dd-bffc-4581-b30d-e9c202c318b7 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:39.599724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 599/1193] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:51:39.604942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ddb1e-65b5-42de-a59b-da4d4ca22d55 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:39.605355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ddb1e-65b5-42de-a59b-da4d4ca22d55 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:39.605533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ddb1e-65b5-42de-a59b-da4d4ca22d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:39.623210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-1e5ddb1e-65b5-42de-a59b-da4d4ca22d55 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:51:39.624397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1e5ddb1e-65b5-42de-a59b-da4d4ca22d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:51:39.691143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 595/1194] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:51:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:51:39.714789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d028dd-bffc-4581-b30d-e9c202c318b7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:39.734160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:39.817823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d028dd-bffc-4581-b30d-e9c202c318b7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:39.850214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:39.850993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:39.851272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:39.851541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:39.861188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:51:39.869340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:51:39.870464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:51:39.871683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 600/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:51:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:51:39.876559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ddb1e-65b5-42de-a59b-da4d4ca22d55 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:39.977980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ddb1e-65b5-42de-a59b-da4d4ca22d55 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:40.004034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:40.011780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:40.012448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:40.012729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:40.013033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:40.013782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:40.014085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:40.015445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:40.024148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 596/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:51:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:40.029964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:40.105988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:40.129135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c520ac-4405-4e0e-9b26-72bea8259d84 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:40.162633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:40.163509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:40.163793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:40.164068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] PATH_INFO: `/v3/users/d83657bfd4e040e7878cf2a1b0c4481d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:40.165697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] RBAC: Authorizing `identity:get_user(user_id=d83657bfd4e040e7878cf2a1b0c4481d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:40.167828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:40.169502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 601/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jun 29 07:51:40 2022] GET /identity/v3/users/d83657bfd4e040e7878cf2a1b0c4481d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:40.176301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:40.276661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16949a1e-651a-487d-a0ab-3b3da85fd9bd None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:40.309741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:40.310398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:40.310679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:40.310954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:40.366215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:40.376234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:40.378372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:40.472511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 597/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:51:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:51:40.478458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:40.576954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b77ed-3c83-44b5-b177-0c57d8f7714b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:40.613005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37473de-84db-4daf-8a40-0940018c178f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:40.613882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37473de-84db-4daf-8a40-0940018c178f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:40.614163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37473de-84db-4daf-8a40-0940018c178f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:40.614436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37473de-84db-4daf-8a40-0940018c178f None admin] PATH_INFO: `/v3/projects/b3dafdff3aa64e819a178591032ded55` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:40.615615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37473de-84db-4daf-8a40-0940018c178f None admin] RBAC: Authorizing `identity:get_project(project_id=b3dafdff3aa64e819a178591032ded55)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:40.617132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37473de-84db-4daf-8a40-0940018c178f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:40.618773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 602/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jun 29 07:51:40 2022] GET /identity/v3/projects/b3dafdff3aa64e819a178591032ded55 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:40.626278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:51:40.715063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:40.725499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f977b-0df7-41e0-b7c3-dd753faecfc1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:51:40.759242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a254b-0517-4f9c-a107-fe574e9de7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:51:40.759899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a254b-0517-4f9c-a107-fe574e9de7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:51:40.760182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a254b-0517-4f9c-a107-fe574e9de7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:51:40.760457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a254b-0517-4f9c-a107-fe574e9de7b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:51:40.761316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a254b-0517-4f9c-a107-fe574e9de7b8 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=b3dafdff3aa64e819a178591032ded55, user.id=d83657bfd4e040e7878cf2a1b0c4481d) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:51:40.761624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a254b-0517-4f9c-a107-fe574e9de7b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:51:40.763054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a254b-0517-4f9c-a107-fe574e9de7b8 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:51:40.768950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 598/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:51:40 2022] GET /identity/v3/role_assignments?user.id=d83657bfd4e040e7878cf2a1b0c4481d&scope.project.id=b3dafdff3aa64e819a178591032ded55&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:51:50.608372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=93 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:51:50.766290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=23 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:38.779817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec49e2-8f43-473c-a5f9-f418e17e9f1e None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:38.780385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec49e2-8f43-473c-a5f9-f418e17e9f1e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:38.780385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec49e2-8f43-473c-a5f9-f418e17e9f1e None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:38.782686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 603/1201] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:52:38.788819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185310-1703-4e4d-a781-a54137df971a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:38.789855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185310-1703-4e4d-a781-a54137df971a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:38.790395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185310-1703-4e4d-a781-a54137df971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:38.817028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fd185310-1703-4e4d-a781-a54137df971a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:52:38.818237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fd185310-1703-4e4d-a781-a54137df971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:52:38.917646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 599/1202] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:52:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:52:38.944146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec49e2-8f43-473c-a5f9-f418e17e9f1e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:39.092748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec49e2-8f43-473c-a5f9-f418e17e9f1e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:39.141050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:39.141947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:39.142363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:39.142607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:39.154150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:39.155673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:39.176955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:39.178707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:39.179703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:39.179703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:39.179703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:39.179703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:39.179703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 604/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:39.184333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd185310-1703-4e4d-a781-a54137df971a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:39.332367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd185310-1703-4e4d-a781-a54137df971a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:39.374473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:39.382573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3ba2c-989b-41c1-bcfc-898b9ff9cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:39.383590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3ba2c-989b-41c1-bcfc-898b9ff9cd68 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:39.384019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3ba2c-989b-41c1-bcfc-898b9ff9cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:39.384345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3ba2c-989b-41c1-bcfc-898b9ff9cd68 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:39.385196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3ba2c-989b-41c1-bcfc-898b9ff9cd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:39.385573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3ba2c-989b-41c1-bcfc-898b9ff9cd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:39.387139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3ba2c-989b-41c1-bcfc-898b9ff9cd68 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:39.409013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 600/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:39.411315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:39.518259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:39.575548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcace785-7354-4279-83b2-2b66c8a51372 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:39.616668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaada0-846a-47a3-b858-249e5c7c7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:39.617327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaada0-846a-47a3-b858-249e5c7c7ad6 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:39.617621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaada0-846a-47a3-b858-249e5c7c7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:39.618151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaada0-846a-47a3-b858-249e5c7c7ad6 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:39.618919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aaada0-846a-47a3-b858-249e5c7c7ad6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:39.620408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aaada0-846a-47a3-b858-249e5c7c7ad6 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:39.648520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 605/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jun 29 07:52:39 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:52:41.980672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f600-16ed-4c83-a69f-03e4f6aeeb9b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:41.981246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f600-16ed-4c83-a69f-03e4f6aeeb9b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:41.981341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f600-16ed-4c83-a69f-03e4f6aeeb9b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:41.985337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 601/1206] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:52:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:52:41.992475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51bc4-680e-4c07-bed4-6e0beac1bbdc None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:41.993015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51bc4-680e-4c07-bed4-6e0beac1bbdc None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:41.993163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51bc4-680e-4c07-bed4-6e0beac1bbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:42.012703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-41d51bc4-680e-4c07-bed4-6e0beac1bbdc None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:52:42.013961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-41d51bc4-680e-4c07-bed4-6e0beac1bbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:52:42.092222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 606/1207] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:52:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:52:42.118429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f600-16ed-4c83-a69f-03e4f6aeeb9b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:42.248151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f600-16ed-4c83-a69f-03e4f6aeeb9b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:42.295543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:42.297179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:42.297959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:42.298331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:42.313659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:52:42.321473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:42.322564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:42.329215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 602/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:52:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:42.331769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d51bc4-680e-4c07-bed4-6e0beac1bbdc None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:42.377403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:42.448860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d51bc4-680e-4c07-bed4-6e0beac1bbdc None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:42.486771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:42.487506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:42.487792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:42.488068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:42.488921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:42.489279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:42.490658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:42.502082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 607/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:52:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:42.506141 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:42.568936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:42.635233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1ac6a-9eeb-48d5-8580-3b11b5fa7d2a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:42.675321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:42.676228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:42.676600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:42.676896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:42.694828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:42.696535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:42.717159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:42.718336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:42.719308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:42.719308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:42.719308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:42.719308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:42.719308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 603/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:42.726050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:42.855391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:42.861004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c423309-91d0-4e5d-b8ab-61c20aff6302 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:42.911203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:42.912042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:42.912399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:42.912511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:42.913285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:42.913676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:42.915385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:42.931326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 608/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:42.938325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:43.082903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffced22-7bfd-48d7-8feb-19cd13329d91 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:43.085265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:43.139975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:43.141698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:43.142135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:43.142416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:43.152479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:43.153997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:52:43.173875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:43.175823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:43.178098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 604/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:43.182396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:43.310109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0381-aec4-4ced-bf98-1c4def423ab4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:43.351608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:43.352396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:43.352711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:43.353045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:43.354085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:43.354408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:43.356047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:43.384780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 609/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jun 29 07:52:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:43.391944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:43.528959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b8ed5-46e0-47ae-9a10-5ee0285d6962 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:43.569188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:43.569789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:43.570072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:43.570414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:43.619736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:43.682164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:43.683541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:43.806046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 605/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:52:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 07:52:43.812748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:43.836847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:43.936751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07e7fb-6587-4c7a-a533-5223d0e9e5cd None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:43.978786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05040da-511b-42c4-8963-017e93629772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:43.979878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05040da-511b-42c4-8963-017e93629772 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:43.980170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05040da-511b-42c4-8963-017e93629772 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:43.980451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05040da-511b-42c4-8963-017e93629772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:43.992376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05040da-511b-42c4-8963-017e93629772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:43.993958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05040da-511b-42c4-8963-017e93629772 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-b05040da-511b-42c4-8963-017e93629772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:44.022385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:44.023577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:44.024517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:44.024517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:44.024517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:44.024517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:44.024517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 610/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:52:44.029678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:44.109759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:44.152380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486523d-28c3-462c-a622-e23d341d4105 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:44.199319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:44.200026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:44.200345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:44.200714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:44.201541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:44.201864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:44.204541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:44.218191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 606/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:52:44.225227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05040da-511b-42c4-8963-017e93629772 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:44.317127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:44.361944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05040da-511b-42c4-8963-017e93629772 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:44.404730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:44.406037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:44.406524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:44.407000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:44.414513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:44.416205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:52:44.432835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:44.433861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:44.435380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 611/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:52:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:44.440357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:44.555541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b4a7b-9850-4a24-b8bd-7936d788cde6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:44.580032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:44.594353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1c79-d3a9-4adf-bbdb-d299b8300e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:44.595296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1c79-d3a9-4adf-bbdb-d299b8300e79 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:44.595993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1c79-d3a9-4adf-bbdb-d299b8300e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:44.596380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1c79-d3a9-4adf-bbdb-d299b8300e79 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:44.597372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1c79-d3a9-4adf-bbdb-d299b8300e79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:44.597887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1c79-d3a9-4adf-bbdb-d299b8300e79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:44.599698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1c79-d3a9-4adf-bbdb-d299b8300e79 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:44.621441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 607/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:52:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:44.623339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:44.737431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e30cff-72ca-4f60-a180-389d975194a1 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:44.768116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:44.774274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e0371-8cf8-4d28-b278-44d7f9cbddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:44.774991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e0371-8cf8-4d28-b278-44d7f9cbddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:44.775310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e0371-8cf8-4d28-b278-44d7f9cbddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:44.775638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e0371-8cf8-4d28-b278-44d7f9cbddf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:44.776694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e0371-8cf8-4d28-b278-44d7f9cbddf1 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=4fdc728b26894afa85073c08c7c34ecb) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:44.777159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e0371-8cf8-4d28-b278-44d7f9cbddf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:44.778548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e0371-8cf8-4d28-b278-44d7f9cbddf1 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:44.797103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 612/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:52:44 2022] GET /identity/v3/role_assignments?user.id=4fdc728b26894afa85073c08c7c34ecb&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:47.157508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d09f2-a0a1-4657-8224-871df1e48771 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:47.158920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d09f2-a0a1-4657-8224-871df1e48771 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:47.159411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d09f2-a0a1-4657-8224-871df1e48771 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:47.161681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 608/1220] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:52:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:52:47.169701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27810338-7da7-40ef-bfb6-1059d9d44499 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:47.170482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27810338-7da7-40ef-bfb6-1059d9d44499 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:47.171094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27810338-7da7-40ef-bfb6-1059d9d44499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:47.202467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-27810338-7da7-40ef-bfb6-1059d9d44499 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:52:47.205516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-27810338-7da7-40ef-bfb6-1059d9d44499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:52:47.321729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 613/1221] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:52:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:52:47.351108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d09f2-a0a1-4657-8224-871df1e48771 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:47.538746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d09f2-a0a1-4657-8224-871df1e48771 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:47.596537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:47.597471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:47.597802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:47.598125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:47.614472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:52:47.630486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:47.632415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:47.634137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 609/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:52:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:47.642669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27810338-7da7-40ef-bfb6-1059d9d44499 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:47.788036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27810338-7da7-40ef-bfb6-1059d9d44499 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:47.829368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:47.842602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:47.843442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:47.843825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:47.844166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:47.845069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] RBAC: Adding query filter params (name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:47.845440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:47.846994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:47.865488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 614/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:52:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:47.866798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:47.929169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:48.017867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c57f8-df72-4fa2-a0cd-27875a6c627a None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:48.068769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:48.069616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:48.069903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:48.070187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:48.082410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:48.083947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:48.105366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:48.106493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:48.107465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:48.107465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:48.107465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:48.107465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:48.112503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 610/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:48.113780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:48.268880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32934db-f8f3-4ec5-b2ef-fa499939e7db None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:48.317235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:48.318117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:48.318515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:48.319019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:48.319839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:48.320266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:48.321959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:48.337958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 615/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:52:48.345446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:48.467647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:48.477869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863da8b-1144-4a66-af3e-0ebed66a50ba None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:48.524912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:48.526005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:48.526340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:48.526779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:48.538284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:48.539908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:52:48.556104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:48.558200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:48.560415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:48.565010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 611/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:48.565250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:48.622449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:48.681767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c01703-fdba-4420-8e56-d5e6feb14ea4 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:48.726330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:48.727157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:48.727635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:48.728114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:48.729667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:48.730251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:48.731951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:48.756760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 616/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jun 29 07:52:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:52:48.762984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:48.894929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9f1d9-70e7-4b4e-9f4a-44f33d360942 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:48.936725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:48.937724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:48.938128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:48.938414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:48.951050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:48.952866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:48.973443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:48.975413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:48.977828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:48.977828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:48.977828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:48.977828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:48.979897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 612/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:48.980898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:49.121418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81cabc-aed7-48d2-b8a1-33a1649cd714 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:49.127525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:49.181301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:49.182783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:49.183344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:49.184153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:49.186067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:49.187341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:49.191058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:49.216529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 617/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:49.225171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:49.242091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:49.392378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42518539-ce72-4f72-9e53-25f7ed3a4136 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:49.443646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:49.444601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:49.444949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:49.445432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:49.456433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:49.458020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:52:49.485129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:49.487013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:49.488876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 613/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:52:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:49.495762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:49.651720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de456b41-7306-4e04-98a7-b3928d10d49f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:49.697171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333b61b-c55c-4255-8f28-d8a7729f054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:49.698390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333b61b-c55c-4255-8f28-d8a7729f054e None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:49.698937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333b61b-c55c-4255-8f28-d8a7729f054e None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:49.699438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333b61b-c55c-4255-8f28-d8a7729f054e None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:49.701065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333b61b-c55c-4255-8f28-d8a7729f054e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:49.701711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333b61b-c55c-4255-8f28-d8a7729f054e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:49.704816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333b61b-c55c-4255-8f28-d8a7729f054e None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:49.741186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 618/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:52:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:52:49.749244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:49.844804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:49.918344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d483b-b4bd-4d5b-9c5b-39b6daac6364 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:49.967351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2f09f-d146-4907-a387-9cf09df8b6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:49.968760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2f09f-d146-4907-a387-9cf09df8b6ee None admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:49.969246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2f09f-d146-4907-a387-9cf09df8b6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:49.969754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2f09f-d146-4907-a387-9cf09df8b6ee None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/4fdc728b26894afa85073c08c7c34ecb/roles/f36cc14a79404ebbaa216562f0f34fa1` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:49.990979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2f09f-d146-4907-a387-9cf09df8b6ee None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=4fdc728b26894afa85073c08c7c34ecb, role_id=f36cc14a79404ebbaa216562f0f34fa1)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:49.993722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2f09f-d146-4907-a387-9cf09df8b6ee None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:50.013015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 614/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:52:49 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/4fdc728b26894afa85073c08c7c34ecb/roles/f36cc14a79404ebbaa216562f0f34fa1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 07:52:52.553607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e468-5322-4c59-92fb-6b81474f6268 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:52.554425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e468-5322-4c59-92fb-6b81474f6268 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:52.554425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e468-5322-4c59-92fb-6b81474f6268 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:52.555785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 619/1233] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:52:52.561878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb7ae7-1423-4d84-a484-896b7e4494a0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:52.562442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb7ae7-1423-4d84-a484-896b7e4494a0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:52.562442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb7ae7-1423-4d84-a484-896b7e4494a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:52.582849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e0bb7ae7-1423-4d84-a484-896b7e4494a0 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:52:52.584859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e0bb7ae7-1423-4d84-a484-896b7e4494a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:52:52.686892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 615/1234] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:52:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:52:52.712663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e468-5322-4c59-92fb-6b81474f6268 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:52.755320 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:52.880191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e468-5322-4c59-92fb-6b81474f6268 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:52.926804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:52.927934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:52.928259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:52.928569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:52.942395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:52:52.960034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:52.961306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 07:52:52.962394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:52.963425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 620/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jun 29 07:52:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:52.970684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb7ae7-1423-4d84-a484-896b7e4494a0 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:53.025605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:53.094156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb7ae7-1423-4d84-a484-896b7e4494a0 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:53.145337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:53.146112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:53.146448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:53.146816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:53.147708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:53.148066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:53.149618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:53.161134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 616/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jun 29 07:52:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:53.168754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:53.307614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:53.334181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ea8-d1c4-43af-8dce-45634eaa5285 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:53.395334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:53.396162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:53.396459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:53.396737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:53.410732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:53.412540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:53.433256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:53.435242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:53.436311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:53.436311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:53.436311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:53.436311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:53.436311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 621/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:53.442468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:53.586825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffbbd4-d976-4094-a703-8bd2ab48bcc9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:53.599125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:53.629955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:53.630639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:53.630918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:53.631189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:53.631931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:53.632232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:53.633691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:53.648307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 617/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:53.654875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:53.815798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131960b-1855-41c5-809e-c31eabf3fca0 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:53.858133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:53.860140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:53.860782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:53.861329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:53.865717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:53.875098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:53.877922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:52:53.903979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:53.906670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:53.907885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 622/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:53.918992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:54.084671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba36d-591c-462a-b553-aa4e6f68d0ec None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:54.132903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:54.133553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:54.133838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:54.134226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:54.135039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:54.135293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:54.136829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:54.160195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 618/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jun 29 07:52:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:54.171532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:54.347213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de493a83-b4c8-4011-b324-27266c159b53 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:54.392216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:54.393280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:54.393783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:54.394350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:54.511044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:54.522250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:54.524748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:54.684059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 623/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:52:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 07:52:54.691210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:54.731998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:54.853236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754557f-4259-497c-b1fe-2d65dd4e75f9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:54.913985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:54.915483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:54.915935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:54.916373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:54.935496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:54.938592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:54.967632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:54.969520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:54.970613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:54.970613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:54.970613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:54.970613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:54.970843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 619/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:52:54.977098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:55.113568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:55.120463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b41fdd-4ddd-4e9c-8b23-03602b7a2ea2 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:55.180249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:55.181131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:55.181486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:55.181854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:55.182677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:55.183195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:55.184857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:55.203704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 624/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:55.210325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:55.347592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:55.389097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ed58b-5b93-4eac-b903-d42803a50a28 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:55.445317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:55.446555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:55.446964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:55.447305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:55.458966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:55.460767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:52:55.481098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:55.483301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:52:55.486683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 620/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:55.495121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:55.656566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6084393-f55d-4ce0-b410-37849404fb4a None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:55.702968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d46e0a-6df5-4eb4-b235-1263c1ec09a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:55.704040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d46e0a-6df5-4eb4-b235-1263c1ec09a8 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:55.704392 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d46e0a-6df5-4eb4-b235-1263c1ec09a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:55.704745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d46e0a-6df5-4eb4-b235-1263c1ec09a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:55.705808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d46e0a-6df5-4eb4-b235-1263c1ec09a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:55.706212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d46e0a-6df5-4eb4-b235-1263c1ec09a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:55.707880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d46e0a-6df5-4eb4-b235-1263c1ec09a8 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:55.733133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 625/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:55.737187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:55.878663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b78323-ef6f-4b9b-bf10-1c4f679f1473 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:55.920215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71eea1-63de-4813-b8d8-69155fbdc9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:55.921956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71eea1-63de-4813-b8d8-69155fbdc9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:55.922674 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71eea1-63de-4813-b8d8-69155fbdc9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:55.923153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71eea1-63de-4813-b8d8-69155fbdc9f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:55.924744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71eea1-63de-4813-b8d8-69155fbdc9f2 None admin] RBAC: Adding query filter params (role.id=f36cc14a79404ebbaa216562f0f34fa1, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=4fdc728b26894afa85073c08c7c34ecb) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:55.925447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71eea1-63de-4813-b8d8-69155fbdc9f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:55.928578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71eea1-63de-4813-b8d8-69155fbdc9f2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:55.948762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 621/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:52:55 2022] GET /identity/v3/role_assignments?user.id=4fdc728b26894afa85073c08c7c34ecb&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=f36cc14a79404ebbaa216562f0f34fa1 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:58.380494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2703d5-9831-4622-892d-8740b002a186 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:58.381232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2703d5-9831-4622-892d-8740b002a186 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:58.381232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2703d5-9831-4622-892d-8740b002a186 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:58.382492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 626/1247] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:52:58.391471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e55d1-0198-4e8d-84f1-a75392f19713 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:58.392579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e55d1-0198-4e8d-84f1-a75392f19713 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:58.393376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e55d1-0198-4e8d-84f1-a75392f19713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:58.401985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:58.421170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-916e55d1-0198-4e8d-84f1-a75392f19713 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:52:58.422913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-916e55d1-0198-4e8d-84f1-a75392f19713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:52:58.524301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 622/1248] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:52:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:52:58.554723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2703d5-9831-4622-892d-8740b002a186 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:58.577993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:58.722649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2703d5-9831-4622-892d-8740b002a186 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:58.769851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:58.770672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:58.770962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:58.771254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:58.792572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:52:58.801932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:58.804331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:58.806232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 627/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:52:58.812603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e55d1-0198-4e8d-84f1-a75392f19713 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:58.954051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e55d1-0198-4e8d-84f1-a75392f19713 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:58.989435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:58.996797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:58.997754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:58.998208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:58.998742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:59.000853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:59.001260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:59.003249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:59.013930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 623/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:59.020148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:59.136853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:59.175775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262e547-511c-4871-92f8-00cd6ce43839 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:59.219077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:59.220006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:59.220292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:59.220573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:59.233892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:59.235410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:52:59.254142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:59.255417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:59.256897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:52:59.256897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:52:59.256897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:52:59.256897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:59.256897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 628/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:52:59.263208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:59.397183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a7650-9014-479d-b2ac-f4609a2818e5 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:59.449157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:59.450268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:59.450867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:59.451306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:59.452404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:59.452942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:59.454949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:59.471811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 624/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:52:59.479225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:59.632323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c517-f947-4257-b0fe-b5be2c531572 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:59.669447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:59.680887 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:59.681922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:59.682310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:59.682587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:59.695078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:59.696585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:52:59.715533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:52:59.718514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:52:59.719905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 629/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:52:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:52:59.725862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:52:59.811240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:52:59.885960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e13a59-b7d4-4eff-8a23-9f35b8a7f13d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:52:59.937409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:52:59.938226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:52:59.938541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:52:59.939329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:52:59.939713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:52:59.940030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:52:59.941778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:52:59.973417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 625/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jun 29 07:52:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:52:59.975214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:00.118886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53831584-7b4c-4020-8cb3-45c8c1d06025 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:00.163997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:00.165095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:00.165592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:00.166130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:00.270485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:00.286408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:00.288211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:00.438362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 630/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:52:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:53:00.445050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:00.588244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9859b-6ca6-4a92-8626-2c46ac5bba9d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:00.605626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:00.637777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:00.638738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:00.639108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:00.639955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:00.656274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:00.657966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:53:00.682200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:00.684297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:00.686233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:53:00.686233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:53:00.686233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:00.686233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:00.686233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 626/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:53:00.691578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:00.805283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:00.819820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501c1c3-4a19-4c03-985c-13cb700e0771 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:00.861226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:00.862070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:00.862399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:00.862745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:00.863674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:00.864000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:00.865842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:00.884948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 631/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:53:00.886483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:01.046524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3536d-2f2c-4df7-93c3-d446a3128ffe None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:01.095889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:01.096995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:01.097370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:01.097686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:01.106596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:01.108443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:53:01.126929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:01.129636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:01.130728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 627/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:53:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 07:53:01.137610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:01.286702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a5bc-e07d-4b7b-a719-8f7766f3efee None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:01.331919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e48230-a163-47ca-90a0-e6b4a85a0009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:01.332461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48230-a163-47ca-90a0-e6b4a85a0009 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:01.332773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48230-a163-47ca-90a0-e6b4a85a0009 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:01.333055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48230-a163-47ca-90a0-e6b4a85a0009 None admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:01.333897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e48230-a163-47ca-90a0-e6b4a85a0009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:01.334219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e48230-a163-47ca-90a0-e6b4a85a0009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:01.336393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e48230-a163-47ca-90a0-e6b4a85a0009 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:01.353598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 632/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:53:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:53:01.360228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:01.391913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:01.528795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb888782-d6cd-41fa-9cbd-6a7d358cff0f None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:01.567979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4c612-7f78-4723-9032-26e7d4ac62ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:01.568758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c612-7f78-4723-9032-26e7d4ac62ee None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:01.569042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c612-7f78-4723-9032-26e7d4ac62ee None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:01.569322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c612-7f78-4723-9032-26e7d4ac62ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:01.570166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4c612-7f78-4723-9032-26e7d4ac62ee None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=4fdc728b26894afa85073c08c7c34ecb) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:01.570479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4c612-7f78-4723-9032-26e7d4ac62ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:01.571932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4c612-7f78-4723-9032-26e7d4ac62ee None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:01.578787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 628/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:53:01 2022] GET /identity/v3/role_assignments?user.id=4fdc728b26894afa85073c08c7c34ecb&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=fe0d00714612495091430762d114121a => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:03.829597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e17b7-6fb5-4c9c-9bbf-506b28204142 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:03.830216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e17b7-6fb5-4c9c-9bbf-506b28204142 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:03.830216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e17b7-6fb5-4c9c-9bbf-506b28204142 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:03.831519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 633/1261] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:53:03.837218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e55c7-26cc-443c-9cb8-30b06795fc96 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:03.837688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e55c7-26cc-443c-9cb8-30b06795fc96 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:03.837877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e55c7-26cc-443c-9cb8-30b06795fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:03.858228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-415e55c7-26cc-443c-9cb8-30b06795fc96 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:53:03.859979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-415e55c7-26cc-443c-9cb8-30b06795fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:53:03.940255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 629/1262] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:53:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:53:03.964137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e17b7-6fb5-4c9c-9bbf-506b28204142 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:03.977400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:04.075835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e17b7-6fb5-4c9c-9bbf-506b28204142 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:04.117711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:04.118426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:04.118714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:04.118993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:04.130644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:53:04.138350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:04.140327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:53:04.141767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 634/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:53:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:04.147499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e55c7-26cc-443c-9cb8-30b06795fc96 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:04.170544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:04.275312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e55c7-26cc-443c-9cb8-30b06795fc96 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:04.317636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:04.318306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:04.318585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:04.318857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:04.319618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:04.319920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:04.321322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:04.330730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 630/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:53:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:04.338477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:04.466349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:04.522439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65582cb-ba70-4d7b-819c-a988a5724699 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:04.560863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:04.562234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:04.562519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:04.562799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:04.574344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:04.575834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:53:04.595006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:04.596183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:04.597156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:53:04.597156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:53:04.597156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:04.597156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:53:04.597156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 635/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:04.607728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:04.735986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:04.760600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f329ac7-40d4-4aba-91cd-8798e13a7c7d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:04.803992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:04.804905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:04.805246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:04.805531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:04.806413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:04.806802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:04.808339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:04.823533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 631/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:04.829577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:04.950277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad4d0f-2dae-46ac-b0c0-94ddb92ea260 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:04.972513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:04.989310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:04.990055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:04.990346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:04.990619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:04.999881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:05.001497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:53:05.018540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:05.019594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:53:05.020610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 636/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:53:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:05.026177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:05.138062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530729e1-aaf3-4dfb-8124-0a9ca80c4dc9 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:05.176179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:05.176915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:05.177292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:05.177583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:05.178466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:05.178774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:05.180223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:05.191688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:05.203945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 632/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jun 29 07:53:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:53:05.212447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:05.321784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc322d4-8bd7-43a2-b236-245bc9b7643b None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:05.359648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:05.360435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:05.360718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:05.361014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:05.373712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:05.375265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:53:05.394318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:05.395463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:05.396397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:53:05.396397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:53:05.396397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:05.396397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:53:05.396397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 637/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:05.403332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:05.513918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e184fd-aea0-4f20-a18f-ca3477939c8c None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:05.549903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:05.550572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:05.550861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:05.551144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:05.551938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:05.552271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:05.553789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:05.566875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 633/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 07:53:05.572691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:05.605575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:05.709075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26209-7f92-4a91-9d02-8d1d9accaf5f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:05.744921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:05.745747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:05.746032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:05.746308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:05.753946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:05.755481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:53:05.772731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:05.773794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:53:05.774705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 638/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:05.779985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:05.843037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:05.889377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bfb87-5104-4a28-b90c-024c2f0ff28d None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:05.924472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b5241-2891-4dc8-b626-bd1eba915795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:05.925167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b5241-2891-4dc8-b626-bd1eba915795 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:05.925452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b5241-2891-4dc8-b626-bd1eba915795 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:05.925727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b5241-2891-4dc8-b626-bd1eba915795 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:05.926509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b5241-2891-4dc8-b626-bd1eba915795 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:05.926819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b5241-2891-4dc8-b626-bd1eba915795 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:05.928257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b5241-2891-4dc8-b626-bd1eba915795 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:05.942628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 634/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:05.949277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:06.043707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:06.054055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13809653-45e6-4474-a527-99ea4d2b27c7 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:06.095197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f35a5c-69b7-4a58-bf52-c0e82f1194c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:06.096052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f35a5c-69b7-4a58-bf52-c0e82f1194c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:06.096380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f35a5c-69b7-4a58-bf52-c0e82f1194c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:06.096700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f35a5c-69b7-4a58-bf52-c0e82f1194c1 None admin] PATH_INFO: `/v3/projects/de99a06049e847098fd35e509e1c797a/users/4fdc728b26894afa85073c08c7c34ecb/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:06.098822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f35a5c-69b7-4a58-bf52-c0e82f1194c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=de99a06049e847098fd35e509e1c797a, user_id=4fdc728b26894afa85073c08c7c34ecb, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:06.100257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f35a5c-69b7-4a58-bf52-c0e82f1194c1 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:06.113520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 639/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jun 29 07:53:05 2022] PUT /identity/v3/projects/de99a06049e847098fd35e509e1c797a/users/4fdc728b26894afa85073c08c7c34ecb/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:53:08.271924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded00a5-04e6-49a4-8fe0-545895738d0d None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:08.272510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded00a5-04e6-49a4-8fe0-545895738d0d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:08.272510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded00a5-04e6-49a4-8fe0-545895738d0d None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:08.274005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 635/1274] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:53:08.280383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd95a-5ffd-4806-9acb-e1852eb73859 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:08.280892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd95a-5ffd-4806-9acb-e1852eb73859 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:08.281137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd95a-5ffd-4806-9acb-e1852eb73859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:08.303323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8c8cd95a-5ffd-4806-9acb-e1852eb73859 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:53:08.304706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8c8cd95a-5ffd-4806-9acb-e1852eb73859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:53:08.386301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 640/1275] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:53:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:53:08.412331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded00a5-04e6-49a4-8fe0-545895738d0d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:08.533720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded00a5-04e6-49a4-8fe0-545895738d0d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:08.553359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:08.572302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:08.573133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:08.573417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:08.573691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:08.585256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 07:53:08.593512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:08.594750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:08.600613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 636/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jun 29 07:53:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:08.601436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd95a-5ffd-4806-9acb-e1852eb73859 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:08.711193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:08.733794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd95a-5ffd-4806-9acb-e1852eb73859 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:08.771773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:08.772451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:08.772738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:08.773041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:08.773804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:08.774114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:08.775488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:08.785473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 641/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jun 29 07:53:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:08.791795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:08.908671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c123b-5492-47bc-a111-89c9fe73dd60 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:08.947288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:08.948233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:08.948556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:08.948829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:08.961807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:08.963390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:53:08.983410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:08.984819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:08.986042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:53:08.986042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:53:08.986042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:08.986042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:08.986042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 637/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:08.991477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:09.110781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2672309-c195-4f76-b050-a2fd8a971ebf None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:09.152410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:09.153241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:09.153605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:09.153964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:09.154956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:09.155430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:09.157110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:09.176145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 642/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:09.182713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:09.228819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:09.315698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385be0f-c1c9-4aa1-8222-2a942fed2ff1 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:09.354721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:09.355537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:09.355827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:09.356107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:09.365255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:09.366641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 07:53:09.381994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:09.383239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:09.384422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 638/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:53:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:09.390184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:09.394174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:09.501569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ed630-3f02-4a87-b584-f83f38868963 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:09.537519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:09.538202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:09.538492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:09.538773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:09.539618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:09.539931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:09.541414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:09.561637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 643/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jun 29 07:53:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:09.568521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:09.674911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:09.685452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819ffb5-603a-435a-abf2-386486891ad0 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:09.727864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:09.728579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:09.728863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:09.729179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:09.810747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:09.812660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:09.911784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:09.926497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 639/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jun 29 07:53:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 07:53:09.933300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:10.009096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:10.050861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c0a2-2f86-4b2a-be82-da13ea9dbd73 None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:10.092087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:10.094157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:10.094663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:10.095016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:10.110146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:10.111958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 07:53:10.134464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:10.135754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:10.137036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 07:53:10.137036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 07:53:10.137036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 07:53:10.137036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:53:10.137036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 644/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:10.142767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:10.268583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d842a6f-35bf-42bf-91b8-2e96fb8fb7a6 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:10.308049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:10.308707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:10.308998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:10.309298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:10.310025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:10.310329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:10.312080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:10.325125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 640/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jun 29 07:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:10.332023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:10.446374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550d4dc-2ffe-4f1b-8ad0-85857c167a6f None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:10.462159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:10.487408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:10.488406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:10.488869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:10.489373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:10.500166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:10.502000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 07:53:10.518402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:10.519525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: ERROR keystone.server.flask.application Jun 29 07:53:10.520627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 645/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jun 29 07:53:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:10.525653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:10.579746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:10.635176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aaf7c0-2bc0-4a7d-af63-f060d0280a89 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:10.682650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c0d8d-05a6-4d10-973e-d08df8ae1966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:10.683486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c0d8d-05a6-4d10-973e-d08df8ae1966 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:10.683779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c0d8d-05a6-4d10-973e-d08df8ae1966 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:10.684134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c0d8d-05a6-4d10-973e-d08df8ae1966 None admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:10.685017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c0d8d-05a6-4d10-973e-d08df8ae1966 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:10.685335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c0d8d-05a6-4d10-973e-d08df8ae1966 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:10.688265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c0d8d-05a6-4d10-973e-d08df8ae1966 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:10.707436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 641/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jun 29 07:53:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:10.713869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:10.835342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1075b49-7147-4c86-856d-8ec156651bfa None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:10.878510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c399c-ce56-4c1c-94d1-bf308c6c76a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:10.879175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c399c-ce56-4c1c-94d1-bf308c6c76a5 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:10.879471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c399c-ce56-4c1c-94d1-bf308c6c76a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:10.879737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c399c-ce56-4c1c-94d1-bf308c6c76a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:10.880575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c399c-ce56-4c1c-94d1-bf308c6c76a5 None admin] RBAC: Adding query filter params (role.id=fe0d00714612495091430762d114121a, scope.project.id=de99a06049e847098fd35e509e1c797a, user.id=4fdc728b26894afa85073c08c7c34ecb) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:10.880890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c399c-ce56-4c1c-94d1-bf308c6c76a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:10.882374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c399c-ce56-4c1c-94d1-bf308c6c76a5 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:10.888753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 646/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jun 29 07:53:10 2022] GET /identity/v3/role_assignments?user.id=4fdc728b26894afa85073c08c7c34ecb&scope.project.id=de99a06049e847098fd35e509e1c797a&role.id=fe0d00714612495091430762d114121a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:12.986802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6352c4-789c-4dc1-b96f-cf679909f8c6 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:12.987963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6352c4-789c-4dc1-b96f-cf679909f8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:12.994181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6352c4-789c-4dc1-b96f-cf679909f8c6 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:12.995760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 642/1288] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:53:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:53:13.004684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af320516-c7f5-49a5-ac96-7141c4dfcce4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:13.005281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af320516-c7f5-49a5-ac96-7141c4dfcce4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:13.005281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af320516-c7f5-49a5-ac96-7141c4dfcce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:13.026806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-af320516-c7f5-49a5-ac96-7141c4dfcce4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:53:13.028097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-af320516-c7f5-49a5-ac96-7141c4dfcce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:53:13.064084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:13.110935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 647/1289] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:53:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:53:13.135391 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6352c4-789c-4dc1-b96f-cf679909f8c6 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:13.268640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6352c4-789c-4dc1-b96f-cf679909f8c6 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:13.281058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:13.309698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:13.311809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:13.312102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:13.312382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:13.313088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:13.314755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:13.329105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:13.331072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:13.332857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 643/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:53:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:13.336705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af320516-c7f5-49a5-ac96-7141c4dfcce4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:13.454635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af320516-c7f5-49a5-ac96-7141c4dfcce4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:13.495284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43785bf3-6a96-4eae-b54d-94e7dafeb85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:13.495958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43785bf3-6a96-4eae-b54d-94e7dafeb85a None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:13.496239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43785bf3-6a96-4eae-b54d-94e7dafeb85a None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:13.496512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43785bf3-6a96-4eae-b54d-94e7dafeb85a None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:13.497248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43785bf3-6a96-4eae-b54d-94e7dafeb85a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:13.497557 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43785bf3-6a96-4eae-b54d-94e7dafeb85a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:13.499810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43785bf3-6a96-4eae-b54d-94e7dafeb85a None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:13.505286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 648/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jun 29 07:53:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:13.511710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:13.627491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aff623-ab75-4773-93d6-22a1fcff0329 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:13.665801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19f053-48f7-4bff-a60a-7e024790a9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:13.666464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f053-48f7-4bff-a60a-7e024790a9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:13.666747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f053-48f7-4bff-a60a-7e024790a9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:13.667166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f053-48f7-4bff-a60a-7e024790a9d2 None admin] PATH_INFO: `/v3/services` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:13.668084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19f053-48f7-4bff-a60a-7e024790a9d2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:13.668576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19f053-48f7-4bff-a60a-7e024790a9d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:13.670142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19f053-48f7-4bff-a60a-7e024790a9d2 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:13.677072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 644/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jun 29 07:53:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:15.810063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd664-c85f-4eb4-ab7b-683dd68e091c None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:15.810788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd664-c85f-4eb4-ab7b-683dd68e091c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:15.810788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd664-c85f-4eb4-ab7b-683dd68e091c None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:15.812685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 649/1293] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:53:15.818191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb0c0c-3114-45d8-8b34-e40aa7f3d084 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:15.818635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb0c0c-3114-45d8-8b34-e40aa7f3d084 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:15.818761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb0c0c-3114-45d8-8b34-e40aa7f3d084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:15.836784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-48bb0c0c-3114-45d8-8b34-e40aa7f3d084 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:53:15.838168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-48bb0c0c-3114-45d8-8b34-e40aa7f3d084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:53:15.869293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:15.908061 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 645/1294] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:53:15.932202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bd664-c85f-4eb4-ab7b-683dd68e091c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:15.963726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:16.032558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bd664-c85f-4eb4-ab7b-683dd68e091c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:16.066498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71c222-e660-4152-9fc7-ad914720c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:16.067099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71c222-e660-4152-9fc7-ad914720c824 None admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:16.067383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71c222-e660-4152-9fc7-ad914720c824 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:16.067660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71c222-e660-4152-9fc7-ad914720c824 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:16.068376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71c222-e660-4152-9fc7-ad914720c824 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:16.069771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71c222-e660-4152-9fc7-ad914720c824 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:16.083674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 650/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jun 29 07:53:15 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:53:18.151636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2001d-3e75-4762-b294-7cd7e5f5c526 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:18.152196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2001d-3e75-4762-b294-7cd7e5f5c526 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:18.152196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2001d-3e75-4762-b294-7cd7e5f5c526 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:18.153631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 646/1296] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:53:18.159694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb5be3-4619-4ed1-aac0-e64c5de5ab4f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:18.160199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb5be3-4619-4ed1-aac0-e64c5de5ab4f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:18.160292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb5be3-4619-4ed1-aac0-e64c5de5ab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:18.177598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c3bb5be3-4619-4ed1-aac0-e64c5de5ab4f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:53:18.178778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c3bb5be3-4619-4ed1-aac0-e64c5de5ab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:53:18.252278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 651/1297] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:53:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:53:18.275662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2001d-3e75-4762-b294-7cd7e5f5c526 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:18.380877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2001d-3e75-4762-b294-7cd7e5f5c526 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:18.427002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:18.427716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:18.427992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:18.428263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:18.428924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:18.430727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:18.443645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:18.444863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:18.446036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 647/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:53:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:18.450899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb5be3-4619-4ed1-aac0-e64c5de5ab4f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:18.469625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:18.554490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb5be3-4619-4ed1-aac0-e64c5de5ab4f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:18.589545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:18.590214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:18.590501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:18.590780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:18.591492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:18.591795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:18.593475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:18.598918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 652/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jun 29 07:53:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:18.605906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:18.637912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:18.715168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a71fa-9fe7-43e4-ac9f-8f708b185194 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:18.753163 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59f2c5-b879-4da4-830a-35b6b9ce9990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:18.753836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f2c5-b879-4da4-830a-35b6b9ce9990 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:18.754124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f2c5-b879-4da4-830a-35b6b9ce9990 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:18.754406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f2c5-b879-4da4-830a-35b6b9ce9990 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:18.755211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59f2c5-b879-4da4-830a-35b6b9ce9990 None admin] RBAC: Adding query filter params (interface=public, service_id=456809788bd5441c972371b93abcfa8d, region_id=RegionOne) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:18.755530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59f2c5-b879-4da4-830a-35b6b9ce9990 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:18.756929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59f2c5-b879-4da4-830a-35b6b9ce9990 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:18.762681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 648/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jun 29 07:53:18 2022] GET /identity/v3/endpoints?service_id=456809788bd5441c972371b93abcfa8d&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:18.768669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:18.867906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:18.886200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f00d6-08cf-4880-89e7-51c00e48d79c None admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:18.918765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d5418-e965-4634-bb08-0c0c8045dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:18.919424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d5418-e965-4634-bb08-0c0c8045dd52 None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:18.919707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d5418-e965-4634-bb08-0c0c8045dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:18.920496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d5418-e965-4634-bb08-0c0c8045dd52 None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:18.921242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d5418-e965-4634-bb08-0c0c8045dd52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:18.922866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d5418-e965-4634-bb08-0c0c8045dd52 None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:18.929572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 653/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jun 29 07:53:18 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:53:20.963615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83c64e-2ab1-4ca7-b9cf-8ce270308fa8 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:20.964179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83c64e-2ab1-4ca7-b9cf-8ce270308fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:20.964179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83c64e-2ab1-4ca7-b9cf-8ce270308fa8 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:20.965587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 649/1302] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:53:20.972650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768063de-d522-4263-b4c5-a39ae2bfa058 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:20.973133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768063de-d522-4263-b4c5-a39ae2bfa058 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:20.973245 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768063de-d522-4263-b4c5-a39ae2bfa058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:20.990324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-768063de-d522-4263-b4c5-a39ae2bfa058 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:53:20.991519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-768063de-d522-4263-b4c5-a39ae2bfa058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:53:21.063089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 654/1303] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:53:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 07:53:21.087690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83c64e-2ab1-4ca7-b9cf-8ce270308fa8 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:21.184944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:21.193003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83c64e-2ab1-4ca7-b9cf-8ce270308fa8 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:21.226932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:21.227656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:21.227937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:21.228209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:21.228853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:21.230254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 07:53:21.242071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 07:53:21.243329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application with Lock( Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self._enter() Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application return self.creator() Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application created_value = creator( Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: ERROR keystone.server.flask.application Jun 29 07:53:21.244187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 650/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jun 29 07:53:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 07:53:21.250047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768063de-d522-4263-b4c5-a39ae2bfa058 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:21.325257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:21.348727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768063de-d522-4263-b4c5-a39ae2bfa058 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:21.382587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bab161-8d3b-4299-8fe5-8e7d17a543dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:21.383253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bab161-8d3b-4299-8fe5-8e7d17a543dc None admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:21.383539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bab161-8d3b-4299-8fe5-8e7d17a543dc None admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:21.383818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bab161-8d3b-4299-8fe5-8e7d17a543dc None admin] PATH_INFO: `/v3/services` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:21.384527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bab161-8d3b-4299-8fe5-8e7d17a543dc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:53:21.384837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bab161-8d3b-4299-8fe5-8e7d17a543dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:21.386264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bab161-8d3b-4299-8fe5-8e7d17a543dc None admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:21.392191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 655/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jun 29 07:53:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:53:21.398136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:53:21.507163 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c58f19-9587-4682-9f39-872f9b792876 None admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:53:21.540949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaac9cb-61f7-4eec-9988-08ace8c7a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:53:21.541628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaac9cb-61f7-4eec-9988-08ace8c7a698 None admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:53:21.541910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaac9cb-61f7-4eec-9988-08ace8c7a698 None admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:53:21.542186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaac9cb-61f7-4eec-9988-08ace8c7a698 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:53:21.542927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaac9cb-61f7-4eec-9988-08ace8c7a698 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:53:21.544287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaac9cb-61f7-4eec-9988-08ace8c7a698 None admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:53:21.564156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 651/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jun 29 07:53:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:53:31.388940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=30 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:53:31.560698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:27.750760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e241c9-0d1a-4435-a866-ad6be345e8aa None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:27.751302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e241c9-0d1a-4435-a866-ad6be345e8aa None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:27.751544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e241c9-0d1a-4435-a866-ad6be345e8aa None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:27.758711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf221d-b352-47c1-9b0b-d184a942c56c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:27.759226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf221d-b352-47c1-9b0b-d184a942c56c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:27.759324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf221d-b352-47c1-9b0b-d184a942c56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:27.762604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 656/1307] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:27.865402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-6fbf221d-b352-47c1-9b0b-d184a942c56c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:27.866972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6fbf221d-b352-47c1-9b0b-d184a942c56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:27.992313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 652/1308] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:54:28.045486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff128ad-d9bd-4918-84b5-686da053f1c1 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:28.045854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff128ad-d9bd-4918-84b5-686da053f1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:28.046213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff128ad-d9bd-4918-84b5-686da053f1c1 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:28.048254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 657/1309] 10.222.0.43 () {54 vars in 964 bytes} [Wed Jun 29 07:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:28.055493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2bad3-76de-4402-a9c6-365139ebc4f8 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:28.055790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2bad3-76de-4402-a9c6-365139ebc4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:28.056075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2bad3-76de-4402-a9c6-365139ebc4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:28.170427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-09f2bad3-76de-4402-a9c6-365139ebc4f8 None None] MFA Rules not processed for user `45a558f9c7c84a05a90595fda7fccd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:28.172396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-09f2bad3-76de-4402-a9c6-365139ebc4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:28.197993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:28.279798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 653/1310] 10.222.0.43 () {58 vars in 1061 bytes} [Wed Jun 29 07:54:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:28.285755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff128ad-d9bd-4918-84b5-686da053f1c1 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:28.288017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3ff128ad-d9bd-4918-84b5-686da053f1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:28.496319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff128ad-d9bd-4918-84b5-686da053f1c1 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:28.594582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6f304-f9e6-49ff-bb34-e752eb085e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:28.595274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6f304-f9e6-49ff-bb34-e752eb085e8a service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:28.595558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6f304-f9e6-49ff-bb34-e752eb085e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:28.595835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6f304-f9e6-49ff-bb34-e752eb085e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:28.597692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-40f6f304-f9e6-49ff-bb34-e752eb085e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:28.673101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:28.741855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6f304-f9e6-49ff-bb34-e752eb085e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:28.743418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6f304-f9e6-49ff-bb34-e752eb085e8a service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:28.765052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 658/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:31.187568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63c64-ef7c-48bc-8ef5-932935928eb2 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:31.188104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63c64-ef7c-48bc-8ef5-932935928eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:31.188104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63c64-ef7c-48bc-8ef5-932935928eb2 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:31.189661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 654/1312] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:31.196171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f151c-a567-4aa4-a6ce-3b0fb8f8331b None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:31.196729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f151c-a567-4aa4-a6ce-3b0fb8f8331b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:31.196729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f151c-a567-4aa4-a6ce-3b0fb8f8331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:31.287602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0c3f151c-a567-4aa4-a6ce-3b0fb8f8331b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:31.288904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0c3f151c-a567-4aa4-a6ce-3b0fb8f8331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:31.413021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 659/1313] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:31.453372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63c64-ef7c-48bc-8ef5-932935928eb2 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:31.454926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c1f63c64-ef7c-48bc-8ef5-932935928eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:31.577134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63c64-ef7c-48bc-8ef5-932935928eb2 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:31.660204 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:31.663286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d0312-8091-4435-a641-784cf2cb25c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:31.664171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d0312-8091-4435-a641-784cf2cb25c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:31.664638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d0312-8091-4435-a641-784cf2cb25c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:31.664925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d0312-8091-4435-a641-784cf2cb25c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:31.666674 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-757d0312-8091-4435-a641-784cf2cb25c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:31.782039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d0312-8091-4435-a641-784cf2cb25c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:31.783415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d0312-8091-4435-a641-784cf2cb25c7 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:31.805510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 655/1314] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:34.232681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672edd66-99db-4686-8ea8-68fc03b4f1de None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:34.233384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672edd66-99db-4686-8ea8-68fc03b4f1de None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:34.233384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672edd66-99db-4686-8ea8-68fc03b4f1de None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:34.234663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 660/1315] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:54:34.240777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b6d00-158c-4743-b989-7cc25cc91adb None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:34.241303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b6d00-158c-4743-b989-7cc25cc91adb None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:34.241406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b6d00-158c-4743-b989-7cc25cc91adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:34.322947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-752b6d00-158c-4743-b989-7cc25cc91adb None None] MFA Rules not processed for user `a107bd9da4da4971b4c66f789a21033b`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:34.324743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-752b6d00-158c-4743-b989-7cc25cc91adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:34.383447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:34.431746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 656/1316] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:34.486022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cb54f-1bd6-4958-b3e9-e6ef410f750d None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:34.486427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cb54f-1bd6-4958-b3e9-e6ef410f750d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:34.486868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cb54f-1bd6-4958-b3e9-e6ef410f750d None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:34.489575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 661/1317] 10.222.0.43 () {54 vars in 964 bytes} [Wed Jun 29 07:54:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:54:34.496819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a58c5-9280-4864-8010-aafb63976eae None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:34.497142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a58c5-9280-4864-8010-aafb63976eae None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:34.497438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a58c5-9280-4864-8010-aafb63976eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:34.580190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b42a58c5-9280-4864-8010-aafb63976eae None None] MFA Rules not processed for user `45a558f9c7c84a05a90595fda7fccd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:34.581457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b42a58c5-9280-4864-8010-aafb63976eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:34.665694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 657/1318] 10.222.0.43 () {58 vars in 1061 bytes} [Wed Jun 29 07:54:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:34.676761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cb54f-1bd6-4958-b3e9-e6ef410f750d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:34.678416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7c6cb54f-1bd6-4958-b3e9-e6ef410f750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:34.690548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:34.792741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cb54f-1bd6-4958-b3e9-e6ef410f750d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:34.877168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485f55a-a688-44ca-9158-6e9e8ee1a46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:34.877902 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485f55a-a688-44ca-9158-6e9e8ee1a46b service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:34.878229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485f55a-a688-44ca-9158-6e9e8ee1a46b service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:34.878612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485f55a-a688-44ca-9158-6e9e8ee1a46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:34.880419 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3485f55a-a688-44ca-9158-6e9e8ee1a46b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:34.951880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:35.003085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485f55a-a688-44ca-9158-6e9e8ee1a46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:35.004536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485f55a-a688-44ca-9158-6e9e8ee1a46b service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:35.024529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 662/1319] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:38.182289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63ad8-da8c-444b-8c5d-f2f75d6ad618 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:38.183012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63ad8-da8c-444b-8c5d-f2f75d6ad618 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:38.183743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63ad8-da8c-444b-8c5d-f2f75d6ad618 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:38.185271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 658/1320] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:38.191287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65ee3-1321-4c2e-b647-af6705bec65f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:38.191781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65ee3-1321-4c2e-b647-af6705bec65f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:38.191910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65ee3-1321-4c2e-b647-af6705bec65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:38.268587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-fec65ee3-1321-4c2e-b647-af6705bec65f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:38.269858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fec65ee3-1321-4c2e-b647-af6705bec65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:38.380560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 663/1321] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:38.420191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc63ad8-da8c-444b-8c5d-f2f75d6ad618 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:38.421785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3dc63ad8-da8c-444b-8c5d-f2f75d6ad618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:38.474464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:38.533990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc63ad8-da8c-444b-8c5d-f2f75d6ad618 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:38.620949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f359e3-9812-429c-83ad-106045745c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:38.621633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f359e3-9812-429c-83ad-106045745c81 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:38.622075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f359e3-9812-429c-83ad-106045745c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:38.622365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f359e3-9812-429c-83ad-106045745c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:38.624079 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-60f359e3-9812-429c-83ad-106045745c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:38.727605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:38.743750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f359e3-9812-429c-83ad-106045745c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:38.745438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f359e3-9812-429c-83ad-106045745c81 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:38.765657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 659/1322] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:42.278475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35210be0-b74c-474d-8083-859df57c6654 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:42.279176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35210be0-b74c-474d-8083-859df57c6654 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:42.279176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35210be0-b74c-474d-8083-859df57c6654 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:42.280579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 664/1323] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:42.286783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2f94f-2d06-4b36-960a-c2d8d5b14533 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:42.287649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2f94f-2d06-4b36-960a-c2d8d5b14533 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:42.287816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2f94f-2d06-4b36-960a-c2d8d5b14533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:42.399020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9cb2f94f-2d06-4b36-960a-c2d8d5b14533 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:42.401580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9cb2f94f-2d06-4b36-960a-c2d8d5b14533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:42.567321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 660/1324] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:42.632219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35210be0-b74c-474d-8083-859df57c6654 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:42.635241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-35210be0-b74c-474d-8083-859df57c6654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:42.676485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:42.819873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35210be0-b74c-474d-8083-859df57c6654 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:42.960448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb88b-912d-4326-a20f-f06a24fccf5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:42.961604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb88b-912d-4326-a20f-f06a24fccf5a service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:42.962012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb88b-912d-4326-a20f-f06a24fccf5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:42.962362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb88b-912d-4326-a20f-f06a24fccf5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:42.965488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-063fb88b-912d-4326-a20f-f06a24fccf5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:43.068586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:43.141450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb88b-912d-4326-a20f-f06a24fccf5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:43.143854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb88b-912d-4326-a20f-f06a24fccf5a service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:43.179589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 665/1325] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:45.417213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec0d1-3db1-4411-972c-c41d1861330a None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:45.417737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec0d1-3db1-4411-972c-c41d1861330a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:45.417737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec0d1-3db1-4411-972c-c41d1861330a None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:45.419112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 661/1326] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:45.428216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc241-7bf3-4fc7-a84f-bb53b30f7826 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:45.428802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc241-7bf3-4fc7-a84f-bb53b30f7826 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:45.428802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc241-7bf3-4fc7-a84f-bb53b30f7826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:45.508173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-01acc241-7bf3-4fc7-a84f-bb53b30f7826 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:45.509434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-01acc241-7bf3-4fc7-a84f-bb53b30f7826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:45.618634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 666/1327] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:45.656275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bec0d1-3db1-4411-972c-c41d1861330a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:45.657859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b9bec0d1-3db1-4411-972c-c41d1861330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:45.724408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:45.772658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bec0d1-3db1-4411-972c-c41d1861330a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:45.850507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a8699-3f4f-43d0-a1d0-1cfce39b141d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:45.851150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8699-3f4f-43d0-a1d0-1cfce39b141d service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:45.851435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8699-3f4f-43d0-a1d0-1cfce39b141d service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:45.851711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8699-3f4f-43d0-a1d0-1cfce39b141d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:45.853398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c56a8699-3f4f-43d0-a1d0-1cfce39b141d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:45.966646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:45.969194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a8699-3f4f-43d0-a1d0-1cfce39b141d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:45.971568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a8699-3f4f-43d0-a1d0-1cfce39b141d service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:45.993851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 662/1328] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:48.140304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230124-c77d-4c14-b50a-b871ee025c73 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:48.140852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230124-c77d-4c14-b50a-b871ee025c73 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:48.140852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230124-c77d-4c14-b50a-b871ee025c73 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:48.142322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 667/1329] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:48.148224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb045-88c1-4e22-8e61-bc9cbb42a223 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:48.148871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb045-88c1-4e22-8e61-bc9cbb42a223 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:48.149194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb045-88c1-4e22-8e61-bc9cbb42a223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:48.232628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d6abb045-88c1-4e22-8e61-bc9cbb42a223 None None] MFA Rules not processed for user `a107bd9da4da4971b4c66f789a21033b`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:48.233882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d6abb045-88c1-4e22-8e61-bc9cbb42a223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:48.337389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 663/1330] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:54:48.377139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230124-c77d-4c14-b50a-b871ee025c73 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:48.379999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-84230124-c77d-4c14-b50a-b871ee025c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:48.420890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:48.503629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230124-c77d-4c14-b50a-b871ee025c73 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:48.581803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3dae7-0526-4f8c-998e-1fed39897830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:48.582457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dae7-0526-4f8c-998e-1fed39897830 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:48.582739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dae7-0526-4f8c-998e-1fed39897830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:48.583016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dae7-0526-4f8c-998e-1fed39897830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:48.584648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-67b3dae7-0526-4f8c-998e-1fed39897830 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:48.665901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:48.702963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3dae7-0526-4f8c-998e-1fed39897830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:48.704402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3dae7-0526-4f8c-998e-1fed39897830 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:48.724455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 668/1331] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:51.106757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f711c-23c6-41a7-b274-684de72f843c None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:51.107333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f711c-23c6-41a7-b274-684de72f843c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:51.107333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f711c-23c6-41a7-b274-684de72f843c None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:51.108727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 664/1332] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:51.114072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5e38-220e-45de-be32-ae5d826c273e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:51.114522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5e38-220e-45de-be32-ae5d826c273e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:51.114633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5e38-220e-45de-be32-ae5d826c273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:51.185462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-440a5e38-220e-45de-be32-ae5d826c273e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:51.186732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-440a5e38-220e-45de-be32-ae5d826c273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:51.284776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 669/1333] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:54:51.321238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f711c-23c6-41a7-b274-684de72f843c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:51.322850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-aa0f711c-23c6-41a7-b274-684de72f843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:51.393746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:51.430453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f711c-23c6-41a7-b274-684de72f843c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:51.504909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25577a5-7898-42bb-8967-f5079c9cfad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:51.505581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25577a5-7898-42bb-8967-f5079c9cfad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:51.505867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25577a5-7898-42bb-8967-f5079c9cfad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:51.506146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25577a5-7898-42bb-8967-f5079c9cfad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:51.507784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a25577a5-7898-42bb-8967-f5079c9cfad3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:51.622978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25577a5-7898-42bb-8967-f5079c9cfad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:51.624435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25577a5-7898-42bb-8967-f5079c9cfad3 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:51.634850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:51.645979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 665/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:54:53.824927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135acc89-331f-4953-9482-59fd2b41eb34 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:53.825511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135acc89-331f-4953-9482-59fd2b41eb34 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:53.825511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135acc89-331f-4953-9482-59fd2b41eb34 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:53.826952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 670/1335] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:54:53.832461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced810-b3f5-4537-bd18-c2434d439ed6 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:53.833006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced810-b3f5-4537-bd18-c2434d439ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:53.833117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced810-b3f5-4537-bd18-c2434d439ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:53.904017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-90ced810-b3f5-4537-bd18-c2434d439ed6 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:53.905329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-90ced810-b3f5-4537-bd18-c2434d439ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:54.009443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 666/1336] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:54.051306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135acc89-331f-4953-9482-59fd2b41eb34 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:54.052894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-135acc89-331f-4953-9482-59fd2b41eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:54.097337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:54.172777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135acc89-331f-4953-9482-59fd2b41eb34 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:54.248838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f90b0-2469-4eaf-8985-4886f75ebdaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:54.249524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f90b0-2469-4eaf-8985-4886f75ebdaf service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:54.249807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f90b0-2469-4eaf-8985-4886f75ebdaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:54.250084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f90b0-2469-4eaf-8985-4886f75ebdaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:54.251918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7b7f90b0-2469-4eaf-8985-4886f75ebdaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:54.342931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:54.370427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f90b0-2469-4eaf-8985-4886f75ebdaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:54.371798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f90b0-2469-4eaf-8985-4886f75ebdaf service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:54.393472 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 671/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 07:54:56.929575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc8af-298b-4865-872d-5e4d688f932b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:56.930207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc8af-298b-4865-872d-5e4d688f932b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:56.930207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc8af-298b-4865-872d-5e4d688f932b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:56.931620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 667/1338] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:54:56.938065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd8f61-a51f-4def-83cd-96e592375ef2 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:56.938632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd8f61-a51f-4def-83cd-96e592375ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:56.938632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd8f61-a51f-4def-83cd-96e592375ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:57.019342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ecdd8f61-a51f-4def-83cd-96e592375ef2 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:54:57.020555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ecdd8f61-a51f-4def-83cd-96e592375ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:57.119783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 672/1339] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:54:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:54:57.156659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc8af-298b-4865-872d-5e4d688f932b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:54:57.158233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bc0dc8af-298b-4865-872d-5e4d688f932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:57.252372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:54:57.283969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc8af-298b-4865-872d-5e4d688f932b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:54:57.361477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2bffe-9be4-4ee2-aa65-6dd8c11d5d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:54:57.362137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2bffe-9be4-4ee2-aa65-6dd8c11d5d88 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:54:57.362423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2bffe-9be4-4ee2-aa65-6dd8c11d5d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:54:57.362701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2bffe-9be4-4ee2-aa65-6dd8c11d5d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:54:57.364363 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-daa2bffe-9be4-4ee2-aa65-6dd8c11d5d88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:54:57.479961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2bffe-9be4-4ee2-aa65-6dd8c11d5d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:54:57.481483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2bffe-9be4-4ee2-aa65-6dd8c11d5d88 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:54:57.501077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 668/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:54:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:01.806577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b108f23-8cee-486a-949d-768d1d764370 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:01.808075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b108f23-8cee-486a-949d-768d1d764370 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:01.808075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b108f23-8cee-486a-949d-768d1d764370 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:01.809809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 673/1341] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:01.816394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bb5d2-5531-4480-b41d-892dae0f60f4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:01.816921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bb5d2-5531-4480-b41d-892dae0f60f4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:01.817078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bb5d2-5531-4480-b41d-892dae0f60f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:01.823643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:01.914159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-eb1bb5d2-5531-4480-b41d-892dae0f60f4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:01.915398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-eb1bb5d2-5531-4480-b41d-892dae0f60f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:02.052462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 669/1342] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:02.096165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b108f23-8cee-486a-949d-768d1d764370 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:02.098533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5b108f23-8cee-486a-949d-768d1d764370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:02.163578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:02.218515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b108f23-8cee-486a-949d-768d1d764370 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:02.222302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86344b-d3c7-47df-a783-d4b48d311058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:02.222953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86344b-d3c7-47df-a783-d4b48d311058 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:02.223241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86344b-d3c7-47df-a783-d4b48d311058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:02.223519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86344b-d3c7-47df-a783-d4b48d311058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:02.233924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86344b-d3c7-47df-a783-d4b48d311058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:02.235372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86344b-d3c7-47df-a783-d4b48d311058 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:02.245904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 674/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:05.114583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd4d2-2ee8-4e73-bd28-898ea7064090 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:05.115263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd4d2-2ee8-4e73-bd28-898ea7064090 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:05.115263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd4d2-2ee8-4e73-bd28-898ea7064090 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:05.116756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 670/1344] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:05.123180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413fdb0-3a9b-4660-9b96-7e1a9ca47452 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:05.123699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413fdb0-3a9b-4660-9b96-7e1a9ca47452 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:05.123900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413fdb0-3a9b-4660-9b96-7e1a9ca47452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:05.153786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c413fdb0-3a9b-4660-9b96-7e1a9ca47452 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:05.155026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c413fdb0-3a9b-4660-9b96-7e1a9ca47452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:05.250925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 675/1345] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:05.290283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd4d2-2ee8-4e73-bd28-898ea7064090 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:05.302306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd4d2-2ee8-4e73-bd28-898ea7064090 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:05.305915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f47fb-1925-46f1-91cd-c8bd1a31bdac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:05.306582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f47fb-1925-46f1-91cd-c8bd1a31bdac service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:05.307009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f47fb-1925-46f1-91cd-c8bd1a31bdac service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:05.307906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f47fb-1925-46f1-91cd-c8bd1a31bdac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:05.318673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f47fb-1925-46f1-91cd-c8bd1a31bdac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:05.320222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f47fb-1925-46f1-91cd-c8bd1a31bdac service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:05.331724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 671/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:09.699971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3fcc-6a03-4e8b-8e37-029fa0cc082b None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:09.700503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3fcc-6a03-4e8b-8e37-029fa0cc082b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:09.700503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3fcc-6a03-4e8b-8e37-029fa0cc082b None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:09.701899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 676/1347] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:09.707477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1125e-8f8a-413b-ae25-df4c862ef148 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:09.707985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1125e-8f8a-413b-ae25-df4c862ef148 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:09.708078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1125e-8f8a-413b-ae25-df4c862ef148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:09.728308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d0e1125e-8f8a-413b-ae25-df4c862ef148 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:09.730085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d0e1125e-8f8a-413b-ae25-df4c862ef148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:09.738088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 672/1348] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:09.778028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3fcc-6a03-4e8b-8e37-029fa0cc082b None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:09.789249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3fcc-6a03-4e8b-8e37-029fa0cc082b None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:09.793721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d1296-ba82-4794-9cf0-a00e1c5582e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:09.794455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1296-ba82-4794-9cf0-a00e1c5582e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:09.794899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1296-ba82-4794-9cf0-a00e1c5582e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:09.795176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1296-ba82-4794-9cf0-a00e1c5582e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:09.805163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d1296-ba82-4794-9cf0-a00e1c5582e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:09.807340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d1296-ba82-4794-9cf0-a00e1c5582e8 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:09.817668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 677/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:15.757460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0503dc2-562a-47aa-a4d8-f95f9ed68c24 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:15.758407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0503dc2-562a-47aa-a4d8-f95f9ed68c24 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:15.758895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0503dc2-562a-47aa-a4d8-f95f9ed68c24 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:15.761756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 673/1350] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 07:55:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:15.767521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372575e-f5d3-43cc-8258-573b6d61f97c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:15.768044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372575e-f5d3-43cc-8258-573b6d61f97c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:15.768129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372575e-f5d3-43cc-8258-573b6d61f97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:15.870799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1372575e-f5d3-43cc-8258-573b6d61f97c None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:15.872519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1372575e-f5d3-43cc-8258-573b6d61f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:15.925423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:16.014647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 678/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 07:55:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:19.722261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=72 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:20.183004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9b468-4717-4c71-a503-ac64f011b61c None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:20.183573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9b468-4717-4c71-a503-ac64f011b61c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:20.184004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9b468-4717-4c71-a503-ac64f011b61c None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:20.185698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 674/1352] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 07:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:20.191435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ceb0d-d239-4b29-9b2b-82adb12c4d03 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:20.192005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ceb0d-d239-4b29-9b2b-82adb12c4d03 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:20.192406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ceb0d-d239-4b29-9b2b-82adb12c4d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:20.218272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-158ceb0d-d239-4b29-9b2b-82adb12c4d03 None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:20.219802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-158ceb0d-d239-4b29-9b2b-82adb12c4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:20.231647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 679/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 07:55:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:21.845962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc8781-c0d5-4e56-9f82-7277d3a83468 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:21.846703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc8781-c0d5-4e56-9f82-7277d3a83468 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:21.847695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc8781-c0d5-4e56-9f82-7277d3a83468 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:21.849691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 675/1354] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 07:55:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:21.856410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc9d8-6b58-4b94-bc4d-073f12ec6796 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:21.856920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc9d8-6b58-4b94-bc4d-073f12ec6796 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:21.857109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc9d8-6b58-4b94-bc4d-073f12ec6796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:21.876053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8e7fc9d8-6b58-4b94-bc4d-073f12ec6796 None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:21.877675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8e7fc9d8-6b58-4b94-bc4d-073f12ec6796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:21.887494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 680/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:25.792561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2b17e-5bb4-44ea-a23f-437b601a908f None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:25.793184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2b17e-5bb4-44ea-a23f-437b601a908f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:25.793184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2b17e-5bb4-44ea-a23f-437b601a908f None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:25.796106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 676/1356] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:25.803257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdb28b-288c-482c-92ae-928a712e901d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:25.803798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdb28b-288c-482c-92ae-928a712e901d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:25.803798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdb28b-288c-482c-92ae-928a712e901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:25.833666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-5acdb28b-288c-482c-92ae-928a712e901d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:25.834876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5acdb28b-288c-482c-92ae-928a712e901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:25.843413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 681/1357] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:55:25.917770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc88e6-a244-4b50-939d-e1dbf82147d1 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:25.918667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc88e6-a244-4b50-939d-e1dbf82147d1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:25.919096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc88e6-a244-4b50-939d-e1dbf82147d1 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:25.920796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 677/1358] 10.222.0.43 () {54 vars in 961 bytes} [Wed Jun 29 07:55:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:25.942853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076a514-8ae1-4db9-a65b-e212bcea72e8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:25.943402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076a514-8ae1-4db9-a65b-e212bcea72e8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:25.943892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076a514-8ae1-4db9-a65b-e212bcea72e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:26.035775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e076a514-8ae1-4db9-a65b-e212bcea72e8 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:26.037901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e076a514-8ae1-4db9-a65b-e212bcea72e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:26.151547 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 682/1359] 10.222.0.43 () {58 vars in 1058 bytes} [Wed Jun 29 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:26.159436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc88e6-a244-4b50-939d-e1dbf82147d1 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:26.179612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc88e6-a244-4b50-939d-e1dbf82147d1 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:26.185494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd4e7e-c1f9-4a99-ac09-5df4559d4ea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:26.186153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd4e7e-c1f9-4a99-ac09-5df4559d4ea9 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:26.186442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd4e7e-c1f9-4a99-ac09-5df4559d4ea9 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:26.186725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd4e7e-c1f9-4a99-ac09-5df4559d4ea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:26.195938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd4e7e-c1f9-4a99-ac09-5df4559d4ea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:26.198442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd4e7e-c1f9-4a99-ac09-5df4559d4ea9 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:26.208780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 678/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:28.726391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5cb7-1b86-481d-8dd8-2b156f3f105f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:28.727202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5cb7-1b86-481d-8dd8-2b156f3f105f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:28.727416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5cb7-1b86-481d-8dd8-2b156f3f105f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:28.729696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 683/1361] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:28.736251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c67ee-896e-4fcf-873f-4f11433ed8f2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:28.737271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c67ee-896e-4fcf-873f-4f11433ed8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:28.737765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c67ee-896e-4fcf-873f-4f11433ed8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:28.763092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bd2c67ee-896e-4fcf-873f-4f11433ed8f2 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:28.764960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bd2c67ee-896e-4fcf-873f-4f11433ed8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:28.776495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 679/1362] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:28.833059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e5cb7-1b86-481d-8dd8-2b156f3f105f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:28.851172 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e5cb7-1b86-481d-8dd8-2b156f3f105f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:28.858954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74277a-208a-4eaf-8353-74bb48f99b7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:28.859765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74277a-208a-4eaf-8353-74bb48f99b7c service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:28.860178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74277a-208a-4eaf-8353-74bb48f99b7c service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:28.860648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74277a-208a-4eaf-8353-74bb48f99b7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:28.871679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74277a-208a-4eaf-8353-74bb48f99b7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:28.873720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74277a-208a-4eaf-8353-74bb48f99b7c service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:28.887247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 684/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:31.808157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982ba91-8313-43e1-aa6e-e67153e13f77 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:31.808708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982ba91-8313-43e1-aa6e-e67153e13f77 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:31.809137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982ba91-8313-43e1-aa6e-e67153e13f77 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:31.812241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 680/1364] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:31.824385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db880a-4f52-4b50-9276-1fa901335858 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:31.825648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db880a-4f52-4b50-9276-1fa901335858 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:31.826301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db880a-4f52-4b50-9276-1fa901335858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:31.850637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-70db880a-4f52-4b50-9276-1fa901335858 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:31.852201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-70db880a-4f52-4b50-9276-1fa901335858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:31.864890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 685/1365] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:31.932318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4a0c0-8b41-4998-a8dd-db8c14a7e604 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:31.932942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4a0c0-8b41-4998-a8dd-db8c14a7e604 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:31.933416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4a0c0-8b41-4998-a8dd-db8c14a7e604 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:31.935368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 681/1366] 10.222.0.43 () {54 vars in 961 bytes} [Wed Jun 29 07:55:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:31.945266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3e90c-cd38-4222-a13a-a949fb70e7a0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:31.945568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3e90c-cd38-4222-a13a-a949fb70e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:31.945811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3e90c-cd38-4222-a13a-a949fb70e7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:31.976349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-02d3e90c-cd38-4222-a13a-a949fb70e7a0 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:31.978372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-02d3e90c-cd38-4222-a13a-a949fb70e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:31.994222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 686/1367] 10.222.0.43 () {58 vars in 1058 bytes} [Wed Jun 29 07:55:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:32.006779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4a0c0-8b41-4998-a8dd-db8c14a7e604 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:32.023788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4a0c0-8b41-4998-a8dd-db8c14a7e604 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:32.031495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb888f0-c6df-4c63-a5f3-ab50ee153756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:32.032297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb888f0-c6df-4c63-a5f3-ab50ee153756 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:32.032673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb888f0-c6df-4c63-a5f3-ab50ee153756 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:32.033138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb888f0-c6df-4c63-a5f3-ab50ee153756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:32.043362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb888f0-c6df-4c63-a5f3-ab50ee153756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:32.044890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb888f0-c6df-4c63-a5f3-ab50ee153756 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:32.056307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 682/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 07:55:34.312269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe827a-b709-4ecf-92c1-e61429083500 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:34.312863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe827a-b709-4ecf-92c1-e61429083500 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:34.313197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe827a-b709-4ecf-92c1-e61429083500 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:34.315716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 687/1369] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:34.323395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee51277-3ffb-45bc-a637-6b19b5d65602 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:34.324107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee51277-3ffb-45bc-a637-6b19b5d65602 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:34.324239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee51277-3ffb-45bc-a637-6b19b5d65602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:34.350103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4ee51277-3ffb-45bc-a637-6b19b5d65602 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:34.351486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4ee51277-3ffb-45bc-a637-6b19b5d65602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:34.365046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 683/1370] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:55:34.431349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe827a-b709-4ecf-92c1-e61429083500 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:34.454386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe827a-b709-4ecf-92c1-e61429083500 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:34.464527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598260a8-88ee-4e72-8358-2d03f0942d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:34.466006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598260a8-88ee-4e72-8358-2d03f0942d78 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:34.466490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598260a8-88ee-4e72-8358-2d03f0942d78 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:34.466965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598260a8-88ee-4e72-8358-2d03f0942d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:34.482256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598260a8-88ee-4e72-8358-2d03f0942d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:34.484402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598260a8-88ee-4e72-8358-2d03f0942d78 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:34.504193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 688/1371] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:35.476769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecea52-2566-4344-abfc-7ee6ce1f45c6 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:35.477777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecea52-2566-4344-abfc-7ee6ce1f45c6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:35.478508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecea52-2566-4344-abfc-7ee6ce1f45c6 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:35.483995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 684/1372] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:55:35.485820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27734c24-461c-44e9-b9d3-a3ab1ddff277 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:35.486612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27734c24-461c-44e9-b9d3-a3ab1ddff277 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:35.487045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27734c24-461c-44e9-b9d3-a3ab1ddff277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:35.509759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-27734c24-461c-44e9-b9d3-a3ab1ddff277 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:35.511064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-27734c24-461c-44e9-b9d3-a3ab1ddff277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:35.521057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 689/1373] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:35.922199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8f7e1-22f6-43ee-b02a-d44241d8a600 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:35.923348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8f7e1-22f6-43ee-b02a-d44241d8a600 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:35.923967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8f7e1-22f6-43ee-b02a-d44241d8a600 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:35.927143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 685/1374] 10.222.0.43 () {54 vars in 963 bytes} [Wed Jun 29 07:55:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:35.935995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93648ecc-60c1-4c5f-9c15-14d087537571 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:35.936691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93648ecc-60c1-4c5f-9c15-14d087537571 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:35.937301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93648ecc-60c1-4c5f-9c15-14d087537571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:35.950219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:36.042855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-93648ecc-60c1-4c5f-9c15-14d087537571 None None] MFA Rules not processed for user `24630ccd20794f19a138a19187348920`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:36.045007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-93648ecc-60c1-4c5f-9c15-14d087537571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:36.166650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 690/1375] 10.222.0.43 () {58 vars in 1060 bytes} [Wed Jun 29 07:55:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:36.174514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8f7e1-22f6-43ee-b02a-d44241d8a600 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:36.187046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8f7e1-22f6-43ee-b02a-d44241d8a600 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:36.190745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d988ee4-4505-4c8c-a492-33feb0e4e0fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:36.191397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d988ee4-4505-4c8c-a492-33feb0e4e0fe service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:36.191686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d988ee4-4505-4c8c-a492-33feb0e4e0fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:36.191960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d988ee4-4505-4c8c-a492-33feb0e4e0fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:36.202119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d988ee4-4505-4c8c-a492-33feb0e4e0fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:36.203560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d988ee4-4505-4c8c-a492-33feb0e4e0fe service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:36.229437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 686/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:36.820938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff082bc-b412-45e8-bf88-52a3f5b10db2 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:36.821516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff082bc-b412-45e8-bf88-52a3f5b10db2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:36.821516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff082bc-b412-45e8-bf88-52a3f5b10db2 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:36.827054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 691/1377] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:36.831256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63be0c1-d9e5-4a84-b3b6-92f5452fb856 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:36.832326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63be0c1-d9e5-4a84-b3b6-92f5452fb856 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:36.833383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63be0c1-d9e5-4a84-b3b6-92f5452fb856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:36.862336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d63be0c1-d9e5-4a84-b3b6-92f5452fb856 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:36.863841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d63be0c1-d9e5-4a84-b3b6-92f5452fb856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:36.874056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 687/1378] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:55:36.919039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff082bc-b412-45e8-bf88-52a3f5b10db2 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:36.940021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff082bc-b412-45e8-bf88-52a3f5b10db2 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:36.946381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06744442-4a6d-4dfd-b1e8-bbf5a00d064e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:36.947230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06744442-4a6d-4dfd-b1e8-bbf5a00d064e service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:36.947564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06744442-4a6d-4dfd-b1e8-bbf5a00d064e service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:36.947847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06744442-4a6d-4dfd-b1e8-bbf5a00d064e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:36.957938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06744442-4a6d-4dfd-b1e8-bbf5a00d064e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:36.959585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06744442-4a6d-4dfd-b1e8-bbf5a00d064e service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:36.971636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 692/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:39.146860 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8d09f-b2b0-4650-9072-23bd0aa3cfdb None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:39.147471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8d09f-b2b0-4650-9072-23bd0aa3cfdb None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:39.147471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8d09f-b2b0-4650-9072-23bd0aa3cfdb None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:39.148823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 688/1380] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:39.154528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8a55d-36da-4ec5-ba6a-bb56ee870f81 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:39.155262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8a55d-36da-4ec5-ba6a-bb56ee870f81 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:39.155262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8a55d-36da-4ec5-ba6a-bb56ee870f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:39.177591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2dc8a55d-36da-4ec5-ba6a-bb56ee870f81 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:39.178997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2dc8a55d-36da-4ec5-ba6a-bb56ee870f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:39.190230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 693/1381] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:39.237637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8d09f-b2b0-4650-9072-23bd0aa3cfdb None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:39.253824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8d09f-b2b0-4650-9072-23bd0aa3cfdb None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:39.260320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd891d-40c6-47fc-a52b-c0be57118d3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:39.261094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd891d-40c6-47fc-a52b-c0be57118d3b service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:39.261406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd891d-40c6-47fc-a52b-c0be57118d3b service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:39.261688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd891d-40c6-47fc-a52b-c0be57118d3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:39.271708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd891d-40c6-47fc-a52b-c0be57118d3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:39.273359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd891d-40c6-47fc-a52b-c0be57118d3b service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:39.284188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 689/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:41.829870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b78bc-d15f-43dc-83d3-fffeba74ddff None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:41.830725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b78bc-d15f-43dc-83d3-fffeba74ddff None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:41.830725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b78bc-d15f-43dc-83d3-fffeba74ddff None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:41.833177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 694/1383] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:41.838925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f04b-fa41-43f2-b48c-6931adc506ba None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:41.839870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f04b-fa41-43f2-b48c-6931adc506ba None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:41.840187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f04b-fa41-43f2-b48c-6931adc506ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:41.861805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-6af6f04b-fa41-43f2-b48c-6931adc506ba None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:41.863056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6af6f04b-fa41-43f2-b48c-6931adc506ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:41.872917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 690/1384] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:41.929997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b78bc-d15f-43dc-83d3-fffeba74ddff None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:41.950690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b78bc-d15f-43dc-83d3-fffeba74ddff None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:41.960331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff675a-d11b-4149-8e27-205a746ae046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:41.961533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff675a-d11b-4149-8e27-205a746ae046 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:41.962034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff675a-d11b-4149-8e27-205a746ae046 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:41.963001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff675a-d11b-4149-8e27-205a746ae046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:41.979491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff675a-d11b-4149-8e27-205a746ae046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:41.983064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff675a-d11b-4149-8e27-205a746ae046 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:42.002924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 695/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:44.377393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20035dfa-02d7-4aa8-acc6-cf4a3ca49caa None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:44.378531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20035dfa-02d7-4aa8-acc6-cf4a3ca49caa None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:44.379073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20035dfa-02d7-4aa8-acc6-cf4a3ca49caa None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:44.382499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 691/1386] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:44.388724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b23ce-527c-42d8-acda-f365b6f5c110 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:44.389449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b23ce-527c-42d8-acda-f365b6f5c110 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:44.390028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b23ce-527c-42d8-acda-f365b6f5c110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:44.420007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-780b23ce-527c-42d8-acda-f365b6f5c110 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:44.422099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-780b23ce-527c-42d8-acda-f365b6f5c110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:44.435975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 696/1387] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:55:44.510500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20035dfa-02d7-4aa8-acc6-cf4a3ca49caa None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:44.535412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20035dfa-02d7-4aa8-acc6-cf4a3ca49caa None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:44.544781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3632801-ccd8-42c4-8f0a-8f78caf84bc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:44.546118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3632801-ccd8-42c4-8f0a-8f78caf84bc8 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:44.546693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3632801-ccd8-42c4-8f0a-8f78caf84bc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:44.547264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3632801-ccd8-42c4-8f0a-8f78caf84bc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:44.563386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3632801-ccd8-42c4-8f0a-8f78caf84bc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:44.566292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3632801-ccd8-42c4-8f0a-8f78caf84bc8 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:44.583906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 692/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:47.196056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87be90-8048-4ad7-9572-da31a59b500f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:47.196743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87be90-8048-4ad7-9572-da31a59b500f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:47.197145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87be90-8048-4ad7-9572-da31a59b500f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:47.201365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 697/1389] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:47.209462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3dbda-19be-4844-bde2-e7e2bd0ed0fd None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:47.209970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3dbda-19be-4844-bde2-e7e2bd0ed0fd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:47.210464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3dbda-19be-4844-bde2-e7e2bd0ed0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:47.235785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c2a3dbda-19be-4844-bde2-e7e2bd0ed0fd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:47.237106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c2a3dbda-19be-4844-bde2-e7e2bd0ed0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:47.253027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 693/1390] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:47.328138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87be90-8048-4ad7-9572-da31a59b500f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:47.344369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87be90-8048-4ad7-9572-da31a59b500f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:47.350519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3384b5a-c709-4fea-857f-7b2673759994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:47.351236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384b5a-c709-4fea-857f-7b2673759994 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:47.351452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384b5a-c709-4fea-857f-7b2673759994 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:47.351734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384b5a-c709-4fea-857f-7b2673759994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:47.361459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3384b5a-c709-4fea-857f-7b2673759994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:47.363137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3384b5a-c709-4fea-857f-7b2673759994 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:47.376161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 698/1391] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 07:55:49.732446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38a400-5492-4e7c-b9ee-0ed01828a676 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:49.733040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38a400-5492-4e7c-b9ee-0ed01828a676 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:49.733040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38a400-5492-4e7c-b9ee-0ed01828a676 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:49.734401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 694/1392] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:49.740464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77423ab-9fe2-4e95-b63f-459babae26a5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:49.742026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77423ab-9fe2-4e95-b63f-459babae26a5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:49.742026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77423ab-9fe2-4e95-b63f-459babae26a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:49.764290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f77423ab-9fe2-4e95-b63f-459babae26a5 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:49.766441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f77423ab-9fe2-4e95-b63f-459babae26a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:49.777267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 699/1393] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:49.824859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38a400-5492-4e7c-b9ee-0ed01828a676 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:49.833772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=46 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:49.844139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38a400-5492-4e7c-b9ee-0ed01828a676 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:49.852380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88b8a0-77fb-4e8d-8bf2-b24167472487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:49.854530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88b8a0-77fb-4e8d-8bf2-b24167472487 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:49.854941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88b8a0-77fb-4e8d-8bf2-b24167472487 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:49.855350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88b8a0-77fb-4e8d-8bf2-b24167472487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:49.869643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88b8a0-77fb-4e8d-8bf2-b24167472487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:49.871626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88b8a0-77fb-4e8d-8bf2-b24167472487 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:49.890611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 695/1394] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:52.074534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74bd8-868b-470c-90f8-76c673d93cf3 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:52.075194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74bd8-868b-470c-90f8-76c673d93cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:52.075194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74bd8-868b-470c-90f8-76c673d93cf3 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:52.076500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 700/1395] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:52.081977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621ac43-4206-4dca-9c28-4ae0cb9da164 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:52.082474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621ac43-4206-4dca-9c28-4ae0cb9da164 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:52.082617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621ac43-4206-4dca-9c28-4ae0cb9da164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:52.101539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3621ac43-4206-4dca-9c28-4ae0cb9da164 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:52.103273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3621ac43-4206-4dca-9c28-4ae0cb9da164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:52.111298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 696/1396] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:52.156940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce74bd8-868b-470c-90f8-76c673d93cf3 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:52.176850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce74bd8-868b-470c-90f8-76c673d93cf3 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:52.184664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7a1d2-a1ea-4404-b209-cf7ecdac55ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:52.185669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a1d2-a1ea-4404-b209-cf7ecdac55ec service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:52.186179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a1d2-a1ea-4404-b209-cf7ecdac55ec service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:52.186623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a1d2-a1ea-4404-b209-cf7ecdac55ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:52.197699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7a1d2-a1ea-4404-b209-cf7ecdac55ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:52.199340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7a1d2-a1ea-4404-b209-cf7ecdac55ec service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:52.210909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 701/1397] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:52.808488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98cddc-f339-42ac-a9ed-6b79ef9c4f0d None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:52.808859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98cddc-f339-42ac-a9ed-6b79ef9c4f0d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:52.809308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98cddc-f339-42ac-a9ed-6b79ef9c4f0d None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:52.810781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 697/1398] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:55:52.819193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c32ac-1871-4344-bc44-9ff5f54d342d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:52.819733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c32ac-1871-4344-bc44-9ff5f54d342d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:52.820150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c32ac-1871-4344-bc44-9ff5f54d342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:52.840391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2c7c32ac-1871-4344-bc44-9ff5f54d342d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:52.841639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2c7c32ac-1871-4344-bc44-9ff5f54d342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:52.850046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 702/1399] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:52.915962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36fe7e-fbdb-450a-9e17-b5acea2a039b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:52.916746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36fe7e-fbdb-450a-9e17-b5acea2a039b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:52.917050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36fe7e-fbdb-450a-9e17-b5acea2a039b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:52.918496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 698/1400] 10.222.0.43 () {54 vars in 962 bytes} [Wed Jun 29 07:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:52.926005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0024d1-dcf9-42cc-b05c-fab3e7eeb0c1 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:52.926506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0024d1-dcf9-42cc-b05c-fab3e7eeb0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:52.926894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0024d1-dcf9-42cc-b05c-fab3e7eeb0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:52.983087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0a0024d1-dcf9-42cc-b05c-fab3e7eeb0c1 None None] MFA Rules not processed for user `30d3144975804e369a023316655be2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:52.984324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0a0024d1-dcf9-42cc-b05c-fab3e7eeb0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:53.024073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:53.059757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 703/1401] 10.222.0.43 () {58 vars in 1059 bytes} [Wed Jun 29 07:55:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:53.067380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36fe7e-fbdb-450a-9e17-b5acea2a039b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:53.078617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36fe7e-fbdb-450a-9e17-b5acea2a039b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:53.082240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961fade-bc02-4773-8677-4695ebd4d583 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:53.082962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fade-bc02-4773-8677-4695ebd4d583 service glance] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:53.083156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fade-bc02-4773-8677-4695ebd4d583 service glance] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:53.083440 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fade-bc02-4773-8677-4695ebd4d583 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:53.093441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961fade-bc02-4773-8677-4695ebd4d583 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:53.094882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961fade-bc02-4773-8677-4695ebd4d583 service glance] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:53.105900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 699/1402] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:55:53.913609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facefb9a-74a6-49d7-bce1-51604b133483 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:53.914422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facefb9a-74a6-49d7-bce1-51604b133483 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:53.914422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facefb9a-74a6-49d7-bce1-51604b133483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:53.995563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-facefb9a-74a6-49d7-bce1-51604b133483 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:53.997323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-facefb9a-74a6-49d7-bce1-51604b133483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:54.067538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 704/1403] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 07:55:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:55:54.115522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb692-40be-4ab3-89b8-0b7d64ffb0af None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:54.116050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb692-40be-4ab3-89b8-0b7d64ffb0af None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:54.116151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb692-40be-4ab3-89b8-0b7d64ffb0af None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:54.117788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 700/1404] 10.222.0.43 () {54 vars in 942 bytes} [Wed Jun 29 07:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:54.125286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82bbdd-1212-4d3f-a5d0-7a320c688e47 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:54.125578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82bbdd-1212-4d3f-a5d0-7a320c688e47 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:54.125858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82bbdd-1212-4d3f-a5d0-7a320c688e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:54.177963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8e82bbdd-1212-4d3f-a5d0-7a320c688e47 None None] MFA Rules not processed for user `a411a835bce0475fae1db6e099dc2605`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:54.179461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8e82bbdd-1212-4d3f-a5d0-7a320c688e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:54.259325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 705/1405] 10.222.0.43 () {58 vars in 1039 bytes} [Wed Jun 29 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:55:54.270034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfb692-40be-4ab3-89b8-0b7d64ffb0af None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:54.289990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfb692-40be-4ab3-89b8-0b7d64ffb0af None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:54.299269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebfdbd-3a84-479e-a7c8-fca71606ee4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:54.299941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebfdbd-3a84-479e-a7c8-fca71606ee4a service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:54.300223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebfdbd-3a84-479e-a7c8-fca71606ee4a service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:54.300504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebfdbd-3a84-479e-a7c8-fca71606ee4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:54.310120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebfdbd-3a84-479e-a7c8-fca71606ee4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:54.311765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebfdbd-3a84-479e-a7c8-fca71606ee4a service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:54.322318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 701/1406] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 07:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 07:55:54.451108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe4ff-54e5-4302-a001-24e3bce727fd None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:54.451479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe4ff-54e5-4302-a001-24e3bce727fd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:54.451963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe4ff-54e5-4302-a001-24e3bce727fd None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:54.455027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 706/1407] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:55:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:55:54.461236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5a6f-e74d-41f4-bd43-9357a2f9785c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:54.461549 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5a6f-e74d-41f4-bd43-9357a2f9785c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:54.461815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5a6f-e74d-41f4-bd43-9357a2f9785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:54.485344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fc9a5a6f-e74d-41f4-bd43-9357a2f9785c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:55:54.489366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fc9a5a6f-e74d-41f4-bd43-9357a2f9785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:55:54.498706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 702/1408] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:55:54.544810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fe4ff-54e5-4302-a001-24e3bce727fd None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:55:54.563360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fe4ff-54e5-4302-a001-24e3bce727fd None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:55:54.572996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9762e-4edc-47de-be49-f755ac2b4ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:55:54.573561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9762e-4edc-47de-be49-f755ac2b4ab8 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:55:54.573995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9762e-4edc-47de-be49-f755ac2b4ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:55:54.574287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9762e-4edc-47de-be49-f755ac2b4ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:55:54.583682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9762e-4edc-47de-be49-f755ac2b4ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:55:54.585480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9762e-4edc-47de-be49-f755ac2b4ab8 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:55:54.591369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:55:54.598271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 707/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:04.478492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=23 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:56:17.448798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5644477-3139-403e-bfe5-4796ebd939d2 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:17.449542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5644477-3139-403e-bfe5-4796ebd939d2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:17.449542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5644477-3139-403e-bfe5-4796ebd939d2 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:17.450787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 703/1410] 10.222.0.43 () {54 vars in 918 bytes} [Wed Jun 29 07:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:56:17.456130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab6519-8dd3-4cb8-ba67-5facca6c77e0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:17.456583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab6519-8dd3-4cb8-ba67-5facca6c77e0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:17.456687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab6519-8dd3-4cb8-ba67-5facca6c77e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:17.475187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7dab6519-8dd3-4cb8-ba67-5facca6c77e0 None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:17.476397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7dab6519-8dd3-4cb8-ba67-5facca6c77e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:17.485750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 708/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Jun 29 07:56:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:17.501124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad33c4-6539-44d9-b350-d0e2eb079f1c None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:17.501446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad33c4-6539-44d9-b350-d0e2eb079f1c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:17.501677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad33c4-6539-44d9-b350-d0e2eb079f1c None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:17.503140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 704/1412] 10.222.0.43 () {54 vars in 918 bytes} [Wed Jun 29 07:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:56:17.508923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95675b1-1eb8-4062-ba2b-1d716dbe8fb5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:17.509480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95675b1-1eb8-4062-ba2b-1d716dbe8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:17.509830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95675b1-1eb8-4062-ba2b-1d716dbe8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:17.528033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c95675b1-1eb8-4062-ba2b-1d716dbe8fb5 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:17.529192 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c95675b1-1eb8-4062-ba2b-1d716dbe8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:17.538321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 709/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Jun 29 07:56:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:56:20.100224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4468b3-c418-4e45-b8f8-24c759428be0 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:20.100784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4468b3-c418-4e45-b8f8-24c759428be0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:20.100784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4468b3-c418-4e45-b8f8-24c759428be0 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:20.102702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:20.108210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd33600-a416-4c2c-a3f4-ad3c27e64461 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:20.108649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd33600-a416-4c2c-a3f4-ad3c27e64461 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:20.108782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd33600-a416-4c2c-a3f4-ad3c27e64461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:20.128015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-efd33600-a416-4c2c-a3f4-ad3c27e64461 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:20.129250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-efd33600-a416-4c2c-a3f4-ad3c27e64461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:20.137197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:56:20.183202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4468b3-c418-4e45-b8f8-24c759428be0 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:20.198722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4468b3-c418-4e45-b8f8-24c759428be0 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:20.204469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56ba72-63a2-44ec-981b-8d208c5d307b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:20.205152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56ba72-63a2-44ec-981b-8d208c5d307b service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:20.205467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56ba72-63a2-44ec-981b-8d208c5d307b service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:20.205750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56ba72-63a2-44ec-981b-8d208c5d307b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:20.215766 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56ba72-63a2-44ec-981b-8d208c5d307b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:20.217419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56ba72-63a2-44ec-981b-8d208c5d307b service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:20.228564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 706/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:21.193636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc831ad-da41-4b3a-9c99-7db893c659ea None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:21.194236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc831ad-da41-4b3a-9c99-7db893c659ea None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:21.194236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc831ad-da41-4b3a-9c99-7db893c659ea None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:21.196212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:21.201660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76fdb6-4bcb-4708-90b6-79912c28cdee None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:21.202183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76fdb6-4bcb-4708-90b6-79912c28cdee None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:21.202278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76fdb6-4bcb-4708-90b6-79912c28cdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:21.221839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4b76fdb6-4bcb-4708-90b6-79912c28cdee None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:21.223083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4b76fdb6-4bcb-4708-90b6-79912c28cdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:21.232292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:21.399964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc831ad-da41-4b3a-9c99-7db893c659ea None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:21.415391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc831ad-da41-4b3a-9c99-7db893c659ea None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:21.421291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0006e-0c94-4f25-a956-eb1d154a14b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:21.421955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0006e-0c94-4f25-a956-eb1d154a14b7 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:21.422239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0006e-0c94-4f25-a956-eb1d154a14b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:21.422516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0006e-0c94-4f25-a956-eb1d154a14b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:21.431639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0006e-0c94-4f25-a956-eb1d154a14b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:21.433268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0006e-0c94-4f25-a956-eb1d154a14b7 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:21.442896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 712/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 07:56:22.344088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874db59a-9079-4265-93df-4c5165fb350b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:22.344721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874db59a-9079-4265-93df-4c5165fb350b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:22.344721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874db59a-9079-4265-93df-4c5165fb350b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:22.346094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:22.352471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb5f19-9b44-41de-941b-9314b5b74406 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:22.353200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb5f19-9b44-41de-941b-9314b5b74406 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:22.353200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb5f19-9b44-41de-941b-9314b5b74406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:22.371421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-d0cb5f19-9b44-41de-941b-9314b5b74406 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:22.372624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d0cb5f19-9b44-41de-941b-9314b5b74406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:22.380587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:22.533853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874db59a-9079-4265-93df-4c5165fb350b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:22.549246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874db59a-9079-4265-93df-4c5165fb350b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:22.554619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7febec-d5c7-421d-9d84-a15acad21dc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:22.556070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7febec-d5c7-421d-9d84-a15acad21dc6 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:22.556532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7febec-d5c7-421d-9d84-a15acad21dc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:22.556809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7febec-d5c7-421d-9d84-a15acad21dc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:22.565258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7febec-d5c7-421d-9d84-a15acad21dc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:22.566835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7febec-d5c7-421d-9d84-a15acad21dc6 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:22.577713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 709/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:23.547728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e0446-6101-428b-b1ba-0e3be92dbefd None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:23.548542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e0446-6101-428b-b1ba-0e3be92dbefd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:23.548542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e0446-6101-428b-b1ba-0e3be92dbefd None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:23.550001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:23.555672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94e87-89e5-44a7-a1e1-d3d984820e4d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:23.555672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94e87-89e5-44a7-a1e1-d3d984820e4d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:23.556227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94e87-89e5-44a7-a1e1-d3d984820e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:23.575260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-48b94e87-89e5-44a7-a1e1-d3d984820e4d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:23.576480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-48b94e87-89e5-44a7-a1e1-d3d984820e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:23.585382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:23.741677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e0446-6101-428b-b1ba-0e3be92dbefd None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:23.756109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e0446-6101-428b-b1ba-0e3be92dbefd None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:23.762844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970efd0-ede1-4a27-bba9-2137a861767b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:23.763478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970efd0-ede1-4a27-bba9-2137a861767b service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:23.763761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970efd0-ede1-4a27-bba9-2137a861767b service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:23.764035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970efd0-ede1-4a27-bba9-2137a861767b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:23.772728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970efd0-ede1-4a27-bba9-2137a861767b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:23.774943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970efd0-ede1-4a27-bba9-2137a861767b service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:23.785070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 715/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:24.673165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c0c93-3ee1-4c1b-86c1-5bc5558a7698 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:24.673948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c0c93-3ee1-4c1b-86c1-5bc5558a7698 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:24.673948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c0c93-3ee1-4c1b-86c1-5bc5558a7698 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:24.675280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:56:24.680946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250d96b-234a-4b7b-8a81-d809604a30fd None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:24.681520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250d96b-234a-4b7b-8a81-d809604a30fd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:24.681520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250d96b-234a-4b7b-8a81-d809604a30fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:24.699451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7250d96b-234a-4b7b-8a81-d809604a30fd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:24.700631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7250d96b-234a-4b7b-8a81-d809604a30fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:24.708308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:24.858380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c0c93-3ee1-4c1b-86c1-5bc5558a7698 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:24.878301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c0c93-3ee1-4c1b-86c1-5bc5558a7698 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:24.884483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a1b1-6118-4a7c-986f-11bef1477ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:24.885164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a1b1-6118-4a7c-986f-11bef1477ab7 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:24.885469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a1b1-6118-4a7c-986f-11bef1477ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:24.885734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a1b1-6118-4a7c-986f-11bef1477ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:24.894270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3a1b1-6118-4a7c-986f-11bef1477ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:24.895847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3a1b1-6118-4a7c-986f-11bef1477ab7 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:24.905327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 712/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:25.738250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac62542-451a-4c2a-8001-c18c5e2395ab None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:25.738864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac62542-451a-4c2a-8001-c18c5e2395ab None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:25.738864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac62542-451a-4c2a-8001-c18c5e2395ab None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:25.740209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:25.745955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0adb5-394a-41e6-840a-f7e4dbe18603 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:25.746406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0adb5-394a-41e6-840a-f7e4dbe18603 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:25.746523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0adb5-394a-41e6-840a-f7e4dbe18603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:25.764930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-23c0adb5-394a-41e6-840a-f7e4dbe18603 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:25.766198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-23c0adb5-394a-41e6-840a-f7e4dbe18603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:25.774157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:25.934107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac62542-451a-4c2a-8001-c18c5e2395ab None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:25.948554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac62542-451a-4c2a-8001-c18c5e2395ab None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:25.953898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98fe25-03fd-4dba-84a7-f4ef27806470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:25.954537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98fe25-03fd-4dba-84a7-f4ef27806470 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:25.954820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98fe25-03fd-4dba-84a7-f4ef27806470 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:25.955094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98fe25-03fd-4dba-84a7-f4ef27806470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:25.963391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98fe25-03fd-4dba-84a7-f4ef27806470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:25.964965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98fe25-03fd-4dba-84a7-f4ef27806470 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:25.974720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 718/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:26.809323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915decff-153c-470b-a84b-a8ad3e4aa830 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:26.809902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915decff-153c-470b-a84b-a8ad3e4aa830 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:26.809902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915decff-153c-470b-a84b-a8ad3e4aa830 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:26.811340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:26.817302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16598-44aa-4ed0-a5ab-0971a681146a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:26.819434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16598-44aa-4ed0-a5ab-0971a681146a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:26.819434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16598-44aa-4ed0-a5ab-0971a681146a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:26.839137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-60b16598-44aa-4ed0-a5ab-0971a681146a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:26.840357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-60b16598-44aa-4ed0-a5ab-0971a681146a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:26.848497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:27.004519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915decff-153c-470b-a84b-a8ad3e4aa830 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:27.020393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915decff-153c-470b-a84b-a8ad3e4aa830 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:27.026617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94feaabb-dc93-4bf0-8fab-e47334d4bddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:27.027327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feaabb-dc93-4bf0-8fab-e47334d4bddb service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:27.027755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feaabb-dc93-4bf0-8fab-e47334d4bddb service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:27.028097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feaabb-dc93-4bf0-8fab-e47334d4bddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:27.037138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94feaabb-dc93-4bf0-8fab-e47334d4bddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:27.038824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94feaabb-dc93-4bf0-8fab-e47334d4bddb service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:27.049739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 715/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:27.873858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698cdbc-1e97-4443-8c0a-efe5088d7112 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:27.874444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698cdbc-1e97-4443-8c0a-efe5088d7112 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:27.874444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698cdbc-1e97-4443-8c0a-efe5088d7112 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:27.875851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:27.885214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfafee0-19df-40ba-bf40-959e466c4b74 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:27.885763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfafee0-19df-40ba-bf40-959e466c4b74 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:27.885864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfafee0-19df-40ba-bf40-959e466c4b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:27.908317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fcfafee0-19df-40ba-bf40-959e466c4b74 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:27.909551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fcfafee0-19df-40ba-bf40-959e466c4b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:27.918005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:28.069901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698cdbc-1e97-4443-8c0a-efe5088d7112 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:28.084710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698cdbc-1e97-4443-8c0a-efe5088d7112 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:28.090048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8d94-d042-4d1c-b01e-f3661d08701d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:28.090689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8d94-d042-4d1c-b01e-f3661d08701d service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:28.090969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8d94-d042-4d1c-b01e-f3661d08701d service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:28.091302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8d94-d042-4d1c-b01e-f3661d08701d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:28.099811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8d94-d042-4d1c-b01e-f3661d08701d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:28.101436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8d94-d042-4d1c-b01e-f3661d08701d service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:28.112185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 721/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 07:56:28.962428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a262fc-6318-462f-bb47-f2d904c88f10 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:28.963021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a262fc-6318-462f-bb47-f2d904c88f10 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:28.963021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a262fc-6318-462f-bb47-f2d904c88f10 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:28.964378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 07:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:28.970332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17d02-025e-4a8d-a9ca-c6c72a3f78f0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:28.970794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17d02-025e-4a8d-a9ca-c6c72a3f78f0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:28.970920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17d02-025e-4a8d-a9ca-c6c72a3f78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:28.997313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0bb17d02-025e-4a8d-a9ca-c6c72a3f78f0 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:28.998665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0bb17d02-025e-4a8d-a9ca-c6c72a3f78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:29.010841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 07:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:29.172435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a262fc-6318-462f-bb47-f2d904c88f10 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:29.188411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a262fc-6318-462f-bb47-f2d904c88f10 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:29.193827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de51f5-79b1-4a14-9127-7125e0307b0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:29.194476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de51f5-79b1-4a14-9127-7125e0307b0b service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:29.194765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de51f5-79b1-4a14-9127-7125e0307b0b service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:29.195049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de51f5-79b1-4a14-9127-7125e0307b0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:29.203976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de51f5-79b1-4a14-9127-7125e0307b0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:29.205612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de51f5-79b1-4a14-9127-7125e0307b0b service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:29.216258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 718/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:31.206019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485300-5479-430b-b584-e4923415cdcc None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:31.206567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485300-5479-430b-b584-e4923415cdcc None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:31.206690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485300-5479-430b-b584-e4923415cdcc None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:31.207996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:31.214567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5094b4b-71ee-4c68-8763-e598675f40b4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:31.215037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5094b4b-71ee-4c68-8763-e598675f40b4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:31.215132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5094b4b-71ee-4c68-8763-e598675f40b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:31.234259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e5094b4b-71ee-4c68-8763-e598675f40b4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:31.235515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e5094b4b-71ee-4c68-8763-e598675f40b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:31.243971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:31.283654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f485300-5479-430b-b584-e4923415cdcc None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:31.294967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f485300-5479-430b-b584-e4923415cdcc None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:31.298708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab5b77-9664-4710-a241-5d73cb0197a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:31.299357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab5b77-9664-4710-a241-5d73cb0197a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:31.299640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab5b77-9664-4710-a241-5d73cb0197a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:31.299917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab5b77-9664-4710-a241-5d73cb0197a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:31.310324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab5b77-9664-4710-a241-5d73cb0197a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:31.311778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab5b77-9664-4710-a241-5d73cb0197a9 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:31.323589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 724/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:33.574116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3099225-2665-433a-ab09-0d91f4963128 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:33.574672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3099225-2665-433a-ab09-0d91f4963128 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:33.574672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3099225-2665-433a-ab09-0d91f4963128 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:33.576054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:33.582184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcbf43-3dee-4bdc-b231-61128ea26477 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:33.582666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcbf43-3dee-4bdc-b231-61128ea26477 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:33.583011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcbf43-3dee-4bdc-b231-61128ea26477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:33.602435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6efcbf43-3dee-4bdc-b231-61128ea26477 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:33.603633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6efcbf43-3dee-4bdc-b231-61128ea26477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:33.611578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:33.651625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3099225-2665-433a-ab09-0d91f4963128 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:33.653356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a3099225-2665-433a-ab09-0d91f4963128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:33.666904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3099225-2665-433a-ab09-0d91f4963128 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:33.670403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5f0ad-6ced-4b21-8358-c70ca8fe916c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:33.671040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5f0ad-6ced-4b21-8358-c70ca8fe916c service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:33.671319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5f0ad-6ced-4b21-8358-c70ca8fe916c service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:33.671590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5f0ad-6ced-4b21-8358-c70ca8fe916c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:33.681753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5f0ad-6ced-4b21-8358-c70ca8fe916c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:33.683174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5f0ad-6ced-4b21-8358-c70ca8fe916c service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:33.692683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 721/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:36.068125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8ebef-139e-42e6-bb5b-cfcbe5cf2476 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:36.068661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8ebef-139e-42e6-bb5b-cfcbe5cf2476 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:36.068661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8ebef-139e-42e6-bb5b-cfcbe5cf2476 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:36.070095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:36.075474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e1181-c016-45ea-a0b3-cfc70ffb6814 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:36.075910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e1181-c016-45ea-a0b3-cfc70ffb6814 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:36.076019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e1181-c016-45ea-a0b3-cfc70ffb6814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:36.094645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-260e1181-c016-45ea-a0b3-cfc70ffb6814 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:36.095833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-260e1181-c016-45ea-a0b3-cfc70ffb6814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:36.103780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:36.142516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8ebef-139e-42e6-bb5b-cfcbe5cf2476 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:36.154596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8ebef-139e-42e6-bb5b-cfcbe5cf2476 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:36.158126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f19fe1-c535-4636-9c23-6c90e98d0d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:36.158756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f19fe1-c535-4636-9c23-6c90e98d0d74 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:36.159040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f19fe1-c535-4636-9c23-6c90e98d0d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:36.159311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f19fe1-c535-4636-9c23-6c90e98d0d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:36.168933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f19fe1-c535-4636-9c23-6c90e98d0d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:36.170375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f19fe1-c535-4636-9c23-6c90e98d0d74 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:36.179961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 727/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:38.409432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059464fa-5bc3-4b80-b52b-0c158f867d5b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:38.410052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059464fa-5bc3-4b80-b52b-0c158f867d5b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:38.410052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059464fa-5bc3-4b80-b52b-0c158f867d5b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:38.411414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:38.418143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96d6f-97bf-4941-b4d4-a8304957c8de None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:38.418645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96d6f-97bf-4941-b4d4-a8304957c8de None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:38.418753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96d6f-97bf-4941-b4d4-a8304957c8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:38.438202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-97f96d6f-97bf-4941-b4d4-a8304957c8de None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:38.439423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-97f96d6f-97bf-4941-b4d4-a8304957c8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:38.447859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:38.485975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059464fa-5bc3-4b80-b52b-0c158f867d5b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:38.497992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059464fa-5bc3-4b80-b52b-0c158f867d5b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:38.501546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d0b7b-9aa3-4e64-82d2-6606043352d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:38.502184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d0b7b-9aa3-4e64-82d2-6606043352d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:38.502470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d0b7b-9aa3-4e64-82d2-6606043352d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:38.502746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d0b7b-9aa3-4e64-82d2-6606043352d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:38.512929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d0b7b-9aa3-4e64-82d2-6606043352d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:38.514387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d0b7b-9aa3-4e64-82d2-6606043352d8 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:38.524442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 724/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:40.671174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678929ca-ef7b-4a68-a8b3-cd45a68d4291 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:40.671715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678929ca-ef7b-4a68-a8b3-cd45a68d4291 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:40.671834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678929ca-ef7b-4a68-a8b3-cd45a68d4291 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:40.673199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:40.678868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bede-6e77-4545-886c-2c3dd46ffb64 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:40.679361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bede-6e77-4545-886c-2c3dd46ffb64 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:40.679561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bede-6e77-4545-886c-2c3dd46ffb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:40.698733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f477bede-6e77-4545-886c-2c3dd46ffb64 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:40.699950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f477bede-6e77-4545-886c-2c3dd46ffb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:40.707910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:40.746545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678929ca-ef7b-4a68-a8b3-cd45a68d4291 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:40.757497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678929ca-ef7b-4a68-a8b3-cd45a68d4291 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:40.761348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcd167-bdd9-4d23-9782-e758b7c4c47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:40.761996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd167-bdd9-4d23-9782-e758b7c4c47e service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:40.762276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd167-bdd9-4d23-9782-e758b7c4c47e service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:40.762552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd167-bdd9-4d23-9782-e758b7c4c47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:40.771897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcd167-bdd9-4d23-9782-e758b7c4c47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:40.774550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcd167-bdd9-4d23-9782-e758b7c4c47e service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:40.784300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 730/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:42.949421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3af5a-5992-4434-9f04-8ecc0d075fe9 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:42.949967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3af5a-5992-4434-9f04-8ecc0d075fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:42.949967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3af5a-5992-4434-9f04-8ecc0d075fe9 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:42.951383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:42.956783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe92c5-3fd4-4332-b69e-32f2ab4536af None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:42.957247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe92c5-3fd4-4332-b69e-32f2ab4536af None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:42.957381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe92c5-3fd4-4332-b69e-32f2ab4536af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:42.975883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cbbe92c5-3fd4-4332-b69e-32f2ab4536af None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:42.977674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cbbe92c5-3fd4-4332-b69e-32f2ab4536af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:42.985558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:56:43.024790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3af5a-5992-4434-9f04-8ecc0d075fe9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:43.036881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3af5a-5992-4434-9f04-8ecc0d075fe9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:43.041178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8616c5d-5f24-4a9c-a4ea-7af1092577a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:43.041821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8616c5d-5f24-4a9c-a4ea-7af1092577a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:43.042105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8616c5d-5f24-4a9c-a4ea-7af1092577a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:43.042388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8616c5d-5f24-4a9c-a4ea-7af1092577a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:43.052205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8616c5d-5f24-4a9c-a4ea-7af1092577a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:43.053723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8616c5d-5f24-4a9c-a4ea-7af1092577a1 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:43.064731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 727/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:45.213946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da701a3f-a364-4473-9c5a-d5ca6b3c89a1 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:45.214564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da701a3f-a364-4473-9c5a-d5ca6b3c89a1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:45.214564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da701a3f-a364-4473-9c5a-d5ca6b3c89a1 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:45.215909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:45.221703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cb2e0-96cd-4237-8499-965d08481288 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:45.222128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cb2e0-96cd-4237-8499-965d08481288 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:45.222261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cb2e0-96cd-4237-8499-965d08481288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:45.229183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=70 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:56:45.242242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bc7cb2e0-96cd-4237-8499-965d08481288 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:45.243406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bc7cb2e0-96cd-4237-8499-965d08481288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:45.251614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:45.291704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da701a3f-a364-4473-9c5a-d5ca6b3c89a1 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:45.297302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=73 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:56:45.304326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da701a3f-a364-4473-9c5a-d5ca6b3c89a1 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:45.308003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64b05e-f80b-48b9-b649-8ad27ebd8d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:45.308668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b05e-f80b-48b9-b649-8ad27ebd8d1f service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:45.308941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b05e-f80b-48b9-b649-8ad27ebd8d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:45.309254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b05e-f80b-48b9-b649-8ad27ebd8d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:45.319269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64b05e-f80b-48b9-b649-8ad27ebd8d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:45.321326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64b05e-f80b-48b9-b649-8ad27ebd8d1f service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:45.332652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 733/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:47.482294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6cd7-edeb-4cdc-b7aa-93d00831ee53 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:47.482909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6cd7-edeb-4cdc-b7aa-93d00831ee53 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:47.482909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6cd7-edeb-4cdc-b7aa-93d00831ee53 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:47.484279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:47.490544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050622-1d7c-4a52-afe3-5e5ffa8dc3db None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:47.490544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050622-1d7c-4a52-afe3-5e5ffa8dc3db None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:47.490996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050622-1d7c-4a52-afe3-5e5ffa8dc3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:47.510521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-89050622-1d7c-4a52-afe3-5e5ffa8dc3db None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:47.511758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-89050622-1d7c-4a52-afe3-5e5ffa8dc3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:47.520108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:56:47.560057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6cd7-edeb-4cdc-b7aa-93d00831ee53 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:47.580418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6cd7-edeb-4cdc-b7aa-93d00831ee53 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:47.587170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2aed7-2586-4f32-9da8-3b3bbb8dc3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:47.588308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2aed7-2586-4f32-9da8-3b3bbb8dc3da service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:47.588823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2aed7-2586-4f32-9da8-3b3bbb8dc3da service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:47.589372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2aed7-2586-4f32-9da8-3b3bbb8dc3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:47.601442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2aed7-2586-4f32-9da8-3b3bbb8dc3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:47.602907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2aed7-2586-4f32-9da8-3b3bbb8dc3da service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:47.613837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 730/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:49.735794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1aed-ca89-48f2-bcea-407c54fa0c8f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:49.736361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1aed-ca89-48f2-bcea-407c54fa0c8f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:49.736361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1aed-ca89-48f2-bcea-407c54fa0c8f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:49.737901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:49.744137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76baf8-3703-4575-b86f-1dadfd539447 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:49.744601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76baf8-3703-4575-b86f-1dadfd539447 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:49.744601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76baf8-3703-4575-b86f-1dadfd539447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:49.762879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5f76baf8-3703-4575-b86f-1dadfd539447 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:49.764107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5f76baf8-3703-4575-b86f-1dadfd539447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:49.772175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:56:49.809139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1aed-ca89-48f2-bcea-407c54fa0c8f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:49.821087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1aed-ca89-48f2-bcea-407c54fa0c8f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:49.824752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f35a9-424d-41fb-9b56-93756d046f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:49.825441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f35a9-424d-41fb-9b56-93756d046f75 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:49.825750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f35a9-424d-41fb-9b56-93756d046f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:49.826033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f35a9-424d-41fb-9b56-93756d046f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:49.836491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f35a9-424d-41fb-9b56-93756d046f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:49.837511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f35a9-424d-41fb-9b56-93756d046f75 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:49.847680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 736/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:56:51.943935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6077bb8-5459-4d39-a9b8-c98058aaa783 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:51.944503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6077bb8-5459-4d39-a9b8-c98058aaa783 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:51.944503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6077bb8-5459-4d39-a9b8-c98058aaa783 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:51.946451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 07:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:56:51.951985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17252-64a4-49d1-b7fb-197f624f983b None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:51.952454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17252-64a4-49d1-b7fb-197f624f983b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:51.952570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17252-64a4-49d1-b7fb-197f624f983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:51.970812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-4ee17252-64a4-49d1-b7fb-197f624f983b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:56:51.971998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4ee17252-64a4-49d1-b7fb-197f624f983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:56:51.980288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 07:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 07:56:52.016468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6077bb8-5459-4d39-a9b8-c98058aaa783 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:56:52.027777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6077bb8-5459-4d39-a9b8-c98058aaa783 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:56:52.031635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa9d12-9557-4239-9167-cf550fe808fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:56:52.032275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9d12-9557-4239-9167-cf550fe808fd service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:56:52.032555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9d12-9557-4239-9167-cf550fe808fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:56:52.032840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9d12-9557-4239-9167-cf550fe808fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:56:52.042738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa9d12-9557-4239-9167-cf550fe808fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:56:52.044166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa9d12-9557-4239-9167-cf550fe808fd service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:56:52.054306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 733/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:57:01.965755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=17 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:57:02.049998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=17 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:57:12.402287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29afabd-2067-4922-b15e-8cea2353e611 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:12.402931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29afabd-2067-4922-b15e-8cea2353e611 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:12.402931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29afabd-2067-4922-b15e-8cea2353e611 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:12.404282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 738/1471] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:57:12.409929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b588e-07cf-4538-b00d-9eeb2f39c006 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:12.410449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b588e-07cf-4538-b00d-9eeb2f39c006 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:12.410554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b588e-07cf-4538-b00d-9eeb2f39c006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:12.429912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-dc5b588e-07cf-4538-b00d-9eeb2f39c006 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:12.431105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-dc5b588e-07cf-4538-b00d-9eeb2f39c006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:12.438953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 734/1472] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:12.486769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd191e-3a01-40ce-bb74-1515ebf413b7 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:12.487132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd191e-3a01-40ce-bb74-1515ebf413b7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:12.487420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd191e-3a01-40ce-bb74-1515ebf413b7 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:12.488818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 739/1473] 10.222.0.43 () {54 vars in 962 bytes} [Wed Jun 29 07:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:57:12.494681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf742d2-bb3a-41ef-997e-fac6c657423b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:12.494968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf742d2-bb3a-41ef-997e-fac6c657423b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:12.495285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf742d2-bb3a-41ef-997e-fac6c657423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:12.513248 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bdf742d2-bb3a-41ef-997e-fac6c657423b None None] MFA Rules not processed for user `30d3144975804e369a023316655be2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:12.514458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bdf742d2-bb3a-41ef-997e-fac6c657423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:12.521016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 735/1474] 10.222.0.43 () {58 vars in 1059 bytes} [Wed Jun 29 07:57:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:12.528443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd191e-3a01-40ce-bb74-1515ebf413b7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:57:12.539718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd191e-3a01-40ce-bb74-1515ebf413b7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:57:12.543142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d8eb6-9594-492b-80ef-23bc4696ef05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:57:12.543775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8eb6-9594-492b-80ef-23bc4696ef05 service glance] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:12.544057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8eb6-9594-492b-80ef-23bc4696ef05 service glance] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:12.544335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8eb6-9594-492b-80ef-23bc4696ef05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:12.554306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d8eb6-9594-492b-80ef-23bc4696ef05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:57:12.555863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d8eb6-9594-492b-80ef-23bc4696ef05 service glance] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:57:12.565472 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 740/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:57:14.770185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b2072-e2a1-4e90-9543-ad198d201f32 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:14.770705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b2072-e2a1-4e90-9543-ad198d201f32 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:14.770705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b2072-e2a1-4e90-9543-ad198d201f32 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:14.772062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 736/1476] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:57:14.777761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda297-66d5-4027-be95-c0b1d42a0f49 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:14.778238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda297-66d5-4027-be95-c0b1d42a0f49 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:14.778380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda297-66d5-4027-be95-c0b1d42a0f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:14.796873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-42bda297-66d5-4027-be95-c0b1d42a0f49 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:14.798115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-42bda297-66d5-4027-be95-c0b1d42a0f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:14.806923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 741/1477] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:14.831354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b2072-e2a1-4e90-9543-ad198d201f32 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:57:14.844537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b2072-e2a1-4e90-9543-ad198d201f32 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:57:14.849103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c72ca0-6f25-4d52-af59-618493b84768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:57:14.849757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c72ca0-6f25-4d52-af59-618493b84768 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:14.850045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c72ca0-6f25-4d52-af59-618493b84768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:14.850328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c72ca0-6f25-4d52-af59-618493b84768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:14.852021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c72ca0-6f25-4d52-af59-618493b84768 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:57:14.853880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-05c72ca0-6f25-4d52-af59-618493b84768 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:57:14.853880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 07:57:14.853880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 07:57:14.855211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c72ca0-6f25-4d52-af59-618493b84768 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:57:14.877648 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 737/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jun 29 07:57:14 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:57:16.982212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772f237-25f7-464b-bbe5-846741801f87 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:16.982808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772f237-25f7-464b-bbe5-846741801f87 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:16.982808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772f237-25f7-464b-bbe5-846741801f87 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:16.984157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 742/1479] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:57:16.989501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa90e-5d47-4074-ba86-8a64a514a77d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:16.989951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa90e-5d47-4074-ba86-8a64a514a77d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:16.990084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa90e-5d47-4074-ba86-8a64a514a77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:17.008442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-05baa90e-5d47-4074-ba86-8a64a514a77d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:17.009675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-05baa90e-5d47-4074-ba86-8a64a514a77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:17.017506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 738/1480] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:17.060859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772f237-25f7-464b-bbe5-846741801f87 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:57:17.075274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772f237-25f7-464b-bbe5-846741801f87 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:57:17.080619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3be2e-84ed-4954-aceb-e3630e0365fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:57:17.081299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3be2e-84ed-4954-aceb-e3630e0365fd service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:17.081585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3be2e-84ed-4954-aceb-e3630e0365fd service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:17.081884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3be2e-84ed-4954-aceb-e3630e0365fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:17.090412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3be2e-84ed-4954-aceb-e3630e0365fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:57:17.091995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3be2e-84ed-4954-aceb-e3630e0365fd service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:57:17.102011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 743/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:57:19.339164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0703ab4-e789-49b8-a1c5-b9fc417d8f04 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:19.339733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0703ab4-e789-49b8-a1c5-b9fc417d8f04 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:19.339733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0703ab4-e789-49b8-a1c5-b9fc417d8f04 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:19.341103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 739/1482] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:57:19.346355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846458d-14b5-455d-bcdf-22565c10b3ad None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:19.346784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846458d-14b5-455d-bcdf-22565c10b3ad None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:19.346956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846458d-14b5-455d-bcdf-22565c10b3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:19.365635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f846458d-14b5-455d-bcdf-22565c10b3ad None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:19.367959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f846458d-14b5-455d-bcdf-22565c10b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:19.375831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 744/1483] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:19.411426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0703ab4-e789-49b8-a1c5-b9fc417d8f04 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:57:19.423234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0703ab4-e789-49b8-a1c5-b9fc417d8f04 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:57:19.427005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df70cb-7c1b-4293-b1f0-312c80ee4b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:57:19.427635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df70cb-7c1b-4293-b1f0-312c80ee4b54 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:19.427914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df70cb-7c1b-4293-b1f0-312c80ee4b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:19.428186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df70cb-7c1b-4293-b1f0-312c80ee4b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:19.437345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df70cb-7c1b-4293-b1f0-312c80ee4b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:57:19.438765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df70cb-7c1b-4293-b1f0-312c80ee4b54 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:57:19.448600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 740/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:57:21.536034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0633-5ae4-4de4-8d59-9c0f0a454c3f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:21.536597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0633-5ae4-4de4-8d59-9c0f0a454c3f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:21.536597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0633-5ae4-4de4-8d59-9c0f0a454c3f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:21.538549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 745/1485] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:57:21.543958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04c1cc-3b5f-4727-b6cc-f673ca6b4ba1 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:21.544400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04c1cc-3b5f-4727-b6cc-f673ca6b4ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:21.544509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04c1cc-3b5f-4727-b6cc-f673ca6b4ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:21.563050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-1d04c1cc-3b5f-4727-b6cc-f673ca6b4ba1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:21.564263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1d04c1cc-3b5f-4727-b6cc-f673ca6b4ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:21.572952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 741/1486] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:21.611281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e0633-5ae4-4de4-8d59-9c0f0a454c3f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:57:21.622240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e0633-5ae4-4de4-8d59-9c0f0a454c3f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:57:21.625821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d310185-509e-4bc1-8163-a4ce9c333b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:57:21.626466 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d310185-509e-4bc1-8163-a4ce9c333b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:21.626755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d310185-509e-4bc1-8163-a4ce9c333b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:21.627037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d310185-509e-4bc1-8163-a4ce9c333b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:21.637400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d310185-509e-4bc1-8163-a4ce9c333b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:57:21.638853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d310185-509e-4bc1-8163-a4ce9c333b0f service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:57:21.651428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 746/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:57:23.838743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131e392-e119-416d-bca4-888c317a0b6e None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:23.839315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131e392-e119-416d-bca4-888c317a0b6e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:23.839315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131e392-e119-416d-bca4-888c317a0b6e None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:23.840693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 742/1488] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 07:57:23.846064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a969-3160-428f-ac97-c1ef6621fc2a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:23.846525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a969-3160-428f-ac97-c1ef6621fc2a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:23.846644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a969-3160-428f-ac97-c1ef6621fc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:23.864227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e0a5a969-3160-428f-ac97-c1ef6621fc2a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:23.865474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e0a5a969-3160-428f-ac97-c1ef6621fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:23.875497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 747/1489] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:23.917641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131e392-e119-416d-bca4-888c317a0b6e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:57:23.929920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131e392-e119-416d-bca4-888c317a0b6e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:57:23.933498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925c7ec-671b-4caa-9cdc-d85811782172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:57:23.934135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c7ec-671b-4caa-9cdc-d85811782172 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:23.934419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c7ec-671b-4caa-9cdc-d85811782172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:23.934701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c7ec-671b-4caa-9cdc-d85811782172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:23.943858 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925c7ec-671b-4caa-9cdc-d85811782172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:57:23.946546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925c7ec-671b-4caa-9cdc-d85811782172 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:57:23.956224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 743/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:57:26.238128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600b6ef-ab4e-4669-b1aa-ec4f883e46e3 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:26.238676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600b6ef-ab4e-4669-b1aa-ec4f883e46e3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:26.238676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600b6ef-ab4e-4669-b1aa-ec4f883e46e3 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:26.240076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 748/1491] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jun 29 07:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:57:26.246298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2a2e1-082a-49ab-aa1c-458423bdb972 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:26.246833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2a2e1-082a-49ab-aa1c-458423bdb972 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:26.246931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2a2e1-082a-49ab-aa1c-458423bdb972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:26.265533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f1f2a2e1-082a-49ab-aa1c-458423bdb972 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:57:26.266753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f1f2a2e1-082a-49ab-aa1c-458423bdb972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:57:26.274774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 744/1492] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jun 29 07:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:57:26.314513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600b6ef-ab4e-4669-b1aa-ec4f883e46e3 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:57:26.326433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600b6ef-ab4e-4669-b1aa-ec4f883e46e3 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:57:26.330214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f155-4182-46c6-b893-2dd9ddef0305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:57:26.330864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f155-4182-46c6-b893-2dd9ddef0305 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:57:26.331149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f155-4182-46c6-b893-2dd9ddef0305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:57:26.331434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f155-4182-46c6-b893-2dd9ddef0305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:57:26.340832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543f155-4182-46c6-b893-2dd9ddef0305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:57:26.342298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543f155-4182-46c6-b893-2dd9ddef0305 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:57:26.351679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 749/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:57:36.260257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=33 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:57:36.347501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=33 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:59:01.463358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acc85-fa24-4feb-a666-994fc10e19d9 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.463962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acc85-fa24-4feb-a666-994fc10e19d9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.463962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acc85-fa24-4feb-a666-994fc10e19d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.486063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ff7acc85-fa24-4feb-a666-994fc10e19d9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:59:01.487277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ff7acc85-fa24-4feb-a666-994fc10e19d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:59:01.495742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 745/1494] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 07:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:59:01.504625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb5613-fbcb-47eb-ab27-ab98df87fc19 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.505378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb5613-fbcb-47eb-ab27-ab98df87fc19 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.505378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb5613-fbcb-47eb-ab27-ab98df87fc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.523423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8fdb5613-fbcb-47eb-ab27-ab98df87fc19 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:59:01.524625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8fdb5613-fbcb-47eb-ab27-ab98df87fc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:59:01.532390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 750/1495] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 07:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:59:01.537792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7acc85-fa24-4feb-a666-994fc10e19d9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:01.547397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7acc85-fa24-4feb-a666-994fc10e19d9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:01.553247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:01.553905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.554188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.554465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.555192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 07:59:01.555497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:01.556513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:01.556513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 07:59:01.556513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 07:59:01.557446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:01.570255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 746/1496] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 07:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 07:59:01.578414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb5613-fbcb-47eb-ab27-ab98df87fc19 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:01.587558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb5613-fbcb-47eb-ab27-ab98df87fc19 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:01.592051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:01.592697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.593002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.593293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.593990 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:01.595752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING py.warnings [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:01.595752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: warnings.warn(msg) Jun 29 07:59:01.595752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: Jun 29 07:59:01.597064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:01.634953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 751/1497] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 07:59:01 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:59:01.639140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:01.649395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6ff1-1398-4b48-ab56-36a3304778de admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:01.713594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:01.714182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.714467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.714741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.715468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:01.716467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:01.716467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 07:59:01.716467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 07:59:01.717739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:01.739618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 747/1498] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 07:59:01 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 07:59:01.743752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:01.755060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6246022-4745-475d-8443-90f0c9368021 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:01.759972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:01.760667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.760949 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.761256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.761986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:01.762994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING py.warnings [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:01.762994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: warnings.warn(msg) Jun 29 07:59:01.762994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: Jun 29 07:59:01.763886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:01.777507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 752/1499] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 07:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:59:01.781466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:01.791370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529dc7d-258c-4703-bf0f-8600362c0f74 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:01.795886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc724b-fe3b-4cc8-8ffd-2185015fca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:01.796538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc724b-fe3b-4cc8-8ffd-2185015fca12 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.796825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc724b-fe3b-4cc8-8ffd-2185015fca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.797137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc724b-fe3b-4cc8-8ffd-2185015fca12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.797865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc724b-fe3b-4cc8-8ffd-2185015fca12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:01.798903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-0cbc724b-fe3b-4cc8-8ffd-2185015fca12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:01.798903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 07:59:01.798903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 07:59:01.800040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc724b-fe3b-4cc8-8ffd-2185015fca12 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:01.814125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 748/1500] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 07:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 07:59:01.819045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:01.829140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bface242-883a-46d7-9b6f-2799e50b4696 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:01.834757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb9bba-998e-4656-96ce-62b710d8245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:01.835658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9bba-998e-4656-96ce-62b710d8245a admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.835939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9bba-998e-4656-96ce-62b710d8245a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.836213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9bba-998e-4656-96ce-62b710d8245a admin admin] PATH_INFO: `/v3/projects/0d54f40b9be944d29035b71174239c44/users/eee2691418f045bfb7b4e0bba46fe4b1/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.852563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb9bba-998e-4656-96ce-62b710d8245a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d54f40b9be944d29035b71174239c44, user_id=eee2691418f045bfb7b4e0bba46fe4b1, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:01.853713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING py.warnings [None req-eefb9bba-998e-4656-96ce-62b710d8245a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:01.853713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: warnings.warn(msg) Jun 29 07:59:01.853713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: Jun 29 07:59:01.855687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb9bba-998e-4656-96ce-62b710d8245a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:01.870238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 753/1501] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 07:59:01 2022] PUT /identity/v3/projects/0d54f40b9be944d29035b71174239c44/users/eee2691418f045bfb7b4e0bba46fe4b1/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:59:01.882559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bbfa6-42a2-40ca-b91d-40c0cda624b3 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.882846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bbfa6-42a2-40ca-b91d-40c0cda624b3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.883125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bbfa6-42a2-40ca-b91d-40c0cda624b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.909608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-1a1bbfa6-42a2-40ca-b91d-40c0cda624b3 None None] MFA Rules not processed for user `eee2691418f045bfb7b4e0bba46fe4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:59:01.911635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1a1bbfa6-42a2-40ca-b91d-40c0cda624b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:59:01.977263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 749/1502] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 07:59:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:59:01.990994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04168d3-d15a-4825-9dc2-fa3bd419f61b None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.991277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04168d3-d15a-4825-9dc2-fa3bd419f61b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.991561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04168d3-d15a-4825-9dc2-fa3bd419f61b None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:01.992950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 754/1503] 10.222.0.43 () {54 vars in 963 bytes} [Wed Jun 29 07:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:59:01.997735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f4948-d678-4a2a-ab49-2f724fbd7532 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:01.998027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f4948-d678-4a2a-ab49-2f724fbd7532 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:01.998309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f4948-d678-4a2a-ab49-2f724fbd7532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:02.016239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5e8f4948-d678-4a2a-ab49-2f724fbd7532 None None] MFA Rules not processed for user `24630ccd20794f19a138a19187348920`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:59:02.017510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5e8f4948-d678-4a2a-ab49-2f724fbd7532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:59:02.067625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:59:02.076226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 750/1504] 10.222.0.43 () {58 vars in 1060 bytes} [Wed Jun 29 07:59:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:59:02.083984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04168d3-d15a-4825-9dc2-fa3bd419f61b None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:02.095236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04168d3-d15a-4825-9dc2-fa3bd419f61b None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:02.100048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:02.100690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:02.100990 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:02.101282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:02.109810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:02.111237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:02.124026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 755/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:59:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:59:02.169183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66688f7-ba2f-40fb-afe7-48cf474635cc None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:02.169479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66688f7-ba2f-40fb-afe7-48cf474635cc None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:02.169866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66688f7-ba2f-40fb-afe7-48cf474635cc None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:02.171195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 751/1506] 10.222.0.43 () {52 vars in 829 bytes} [Wed Jun 29 07:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 07:59:02.175059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:02.244278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba852c-dcd3-4507-bc85-97d78ab81338 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:02.248939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:02.249850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:02.250132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:02.250408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] PATH_INFO: `/v3/users/eee2691418f045bfb7b4e0bba46fe4b1` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:02.251606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee2691418f045bfb7b4e0bba46fe4b1)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:02.252624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING py.warnings [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:02.252624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: warnings.warn(msg) Jun 29 07:59:02.252624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: Jun 29 07:59:02.253892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:02.320918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee2691418f045bfb7b4e0bba46fe4b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:02.321582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee2691418f045bfb7b4e0bba46fe4b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:02.334581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee2691418f045bfb7b4e0bba46fe4b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:02.335367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee2691418f045bfb7b4e0bba46fe4b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:02.353554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee2691418f045bfb7b4e0bba46fe4b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:02.357717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:59:02.370409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 756/1507] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 07:59:02 2022] DELETE /identity/v3/users/eee2691418f045bfb7b4e0bba46fe4b1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:59:02.374766 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:02.375053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:02.375330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:02.396284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 07:59:02.398424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:59:02.458914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 752/1508] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 07:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 07:59:02.468355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:02.469787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:59:02.527533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223bc49-0212-4789-817d-f583b5274ba7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:02.531094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a2680-fe08-44bf-a32d-20895502bf25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:02.531744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a2680-fe08-44bf-a32d-20895502bf25 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:02.532028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a2680-fe08-44bf-a32d-20895502bf25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:02.532305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a2680-fe08-44bf-a32d-20895502bf25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:02.542244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a2680-fe08-44bf-a32d-20895502bf25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:02.543672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a2680-fe08-44bf-a32d-20895502bf25 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:02.553482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 757/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 07:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 07:59:02.932517 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 07:59:02.934077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 07:59:02.946870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068a51d-e323-4b33-88eb-b1b11ebef279 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 07:59:02.952277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 07:59:02.953269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 07:59:02.953556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 07:59:02.953839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] PATH_INFO: `/v3/projects/0d54f40b9be944d29035b71174239c44` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 07:59:02.955475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d54f40b9be944d29035b71174239c44)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 07:59:02.956302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 07:59:02.956302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 07:59:02.956302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 07:59:02.957669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 07:59:03.019786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d54f40b9be944d29035b71174239c44', 'request_id': 'req-031fd4f4-6ec2-4033-9c65-aa67cf49a991'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:03.025795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d54f40b9be944d29035b71174239c44', 'request_id': 'req-031fd4f4-6ec2-4033-9c65-aa67cf49a991'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:03.035160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d54f40b9be944d29035b71174239c44', 'request_id': 'req-031fd4f4-6ec2-4033-9c65-aa67cf49a991'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:03.055646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] The token cache is being invalidate because project 0d54f40b9be944d29035b71174239c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 07:59:03.055936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:03.056611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-031fd4f4-6ec2-4033-9c65-aa67cf49a991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 07:59:03.058335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 753/1510] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 07:59:02 2022] DELETE /identity/v3/projects/0d54f40b9be944d29035b71174239c44 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 07:59:12.548868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 07:59:13.052416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:00:42.772065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a224982-7720-4722-8c38-b1403a011baa None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:42.772678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a224982-7720-4722-8c38-b1403a011baa None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:42.772678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a224982-7720-4722-8c38-b1403a011baa None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:42.773972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:00:42.778914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b88a6-bd64-447b-b365-4d1ca9e2c1bd None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:42.779374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b88a6-bd64-447b-b365-4d1ca9e2c1bd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:42.779481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b88a6-bd64-447b-b365-4d1ca9e2c1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:42.797796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e27b88a6-bd64-447b-b365-4d1ca9e2c1bd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:00:42.799024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e27b88a6-bd64-447b-b365-4d1ca9e2c1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:00:42.859645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:00:43.014230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a224982-7720-4722-8c38-b1403a011baa None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:00:43.016047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6a224982-7720-4722-8c38-b1403a011baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:00:43.092845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a224982-7720-4722-8c38-b1403a011baa None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:00:43.098307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdf676-84e2-4c7f-9a04-e369aaa75e37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:00:43.098952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf676-84e2-4c7f-9a04-e369aaa75e37 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:43.099240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf676-84e2-4c7f-9a04-e369aaa75e37 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:43.099517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf676-84e2-4c7f-9a04-e369aaa75e37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:43.107597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdf676-84e2-4c7f-9a04-e369aaa75e37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:00:43.109229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdf676-84e2-4c7f-9a04-e369aaa75e37 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:00:43.118881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 759/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:00:43.955121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097467c2-2212-488c-afd6-d08579fb716e None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:43.955652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097467c2-2212-488c-afd6-d08579fb716e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:43.955652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097467c2-2212-488c-afd6-d08579fb716e None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:43.957668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:00:43.962654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075369-3df4-41a9-939d-41561c1e68b0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:43.963115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075369-3df4-41a9-939d-41561c1e68b0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:43.963502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075369-3df4-41a9-939d-41561c1e68b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:43.982082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9e075369-3df4-41a9-939d-41561c1e68b0 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:00:43.983301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9e075369-3df4-41a9-939d-41561c1e68b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:00:43.991460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:00:44.144477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097467c2-2212-488c-afd6-d08579fb716e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:00:44.158995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097467c2-2212-488c-afd6-d08579fb716e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:00:44.165334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32b9b1-fdca-48dc-a97f-d17e4a41d04a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:00:44.165996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32b9b1-fdca-48dc-a97f-d17e4a41d04a service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:44.166313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32b9b1-fdca-48dc-a97f-d17e4a41d04a service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:44.166595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32b9b1-fdca-48dc-a97f-d17e4a41d04a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:44.174997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32b9b1-fdca-48dc-a97f-d17e4a41d04a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:00:44.178030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32b9b1-fdca-48dc-a97f-d17e4a41d04a service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:00:44.189304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 756/1516] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:00:44.535914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f47ec-9e77-4cc6-82d1-93d60934b880 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:44.536200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f47ec-9e77-4cc6-82d1-93d60934b880 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:44.536485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f47ec-9e77-4cc6-82d1-93d60934b880 None None] PATH_INFO: `/v3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:44.537842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 761/1517] 10.222.0.48 () {54 vars in 925 bytes} [Wed Jun 29 08:00:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:00:44.543998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8761a-ffc0-4d2b-b4cc-19fbba4ad369 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:44.544296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8761a-ffc0-4d2b-b4cc-19fbba4ad369 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:44.544629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8761a-ffc0-4d2b-b4cc-19fbba4ad369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:44.562746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-06b8761a-ffc0-4d2b-b4cc-19fbba4ad369 None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:00:44.563972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-06b8761a-ffc0-4d2b-b4cc-19fbba4ad369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:00:44.626535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 757/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Wed Jun 29 08:00:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:00:53.356957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50800ed-181c-406e-bb9e-54f4bfd12f67 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:53.357518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50800ed-181c-406e-bb9e-54f4bfd12f67 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:53.357518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50800ed-181c-406e-bb9e-54f4bfd12f67 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:53.358845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 762/1519] 10.222.0.43 () {54 vars in 960 bytes} [Wed Jun 29 08:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:00:53.363826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05383-16f8-40bb-aaa2-ed7b9273b1e5 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:53.364271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05383-16f8-40bb-aaa2-ed7b9273b1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:53.364377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05383-16f8-40bb-aaa2-ed7b9273b1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:53.383726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-cbf05383-16f8-40bb-aaa2-ed7b9273b1e5 None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:00:53.384919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-cbf05383-16f8-40bb-aaa2-ed7b9273b1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:00:53.393647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 758/1520] 10.222.0.43 () {58 vars in 1057 bytes} [Wed Jun 29 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:00:53.401549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50800ed-181c-406e-bb9e-54f4bfd12f67 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:00:53.415191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50800ed-181c-406e-bb9e-54f4bfd12f67 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:00:53.420104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512ffe-5e4a-4e46-8c6a-81e1d7669752 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:00:53.420738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512ffe-5e4a-4e46-8c6a-81e1d7669752 service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:53.421076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512ffe-5e4a-4e46-8c6a-81e1d7669752 service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:53.421360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512ffe-5e4a-4e46-8c6a-81e1d7669752 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:53.430749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512ffe-5e4a-4e46-8c6a-81e1d7669752 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:00:53.432352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512ffe-5e4a-4e46-8c6a-81e1d7669752 service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:00:53.443037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 763/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:00:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:00:55.416144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f883efd-7363-46c9-a177-39b0d3003cbf None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:55.416726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f883efd-7363-46c9-a177-39b0d3003cbf None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:55.416726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f883efd-7363-46c9-a177-39b0d3003cbf None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:55.418107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:00:55.423025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77391d0b-89ac-42d8-ace8-44cd94a1d08f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:55.423486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77391d0b-89ac-42d8-ace8-44cd94a1d08f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:55.423583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77391d0b-89ac-42d8-ace8-44cd94a1d08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:55.442491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-77391d0b-89ac-42d8-ace8-44cd94a1d08f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:00:55.443688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-77391d0b-89ac-42d8-ace8-44cd94a1d08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:00:55.452244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:00:55.606237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f883efd-7363-46c9-a177-39b0d3003cbf None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:00:55.621665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f883efd-7363-46c9-a177-39b0d3003cbf None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:00:55.627838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000f332-3f91-4651-b9e9-50a7ed251909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:00:55.628482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000f332-3f91-4651-b9e9-50a7ed251909 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:55.628767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000f332-3f91-4651-b9e9-50a7ed251909 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:55.629083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000f332-3f91-4651-b9e9-50a7ed251909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:55.638583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000f332-3f91-4651-b9e9-50a7ed251909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:00:55.640167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000f332-3f91-4651-b9e9-50a7ed251909 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:00:55.649669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 760/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:00:56.501637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a157cb0-d562-404c-ab86-6544039c59c1 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:56.502183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a157cb0-d562-404c-ab86-6544039c59c1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:56.502183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a157cb0-d562-404c-ab86-6544039c59c1 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:56.503583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:00:56.510234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef24d9b-ccf2-4b36-a24f-b541891e5d2e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:56.510742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef24d9b-ccf2-4b36-a24f-b541891e5d2e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:56.510853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef24d9b-ccf2-4b36-a24f-b541891e5d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:56.529426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9ef24d9b-ccf2-4b36-a24f-b541891e5d2e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:00:56.530636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9ef24d9b-ccf2-4b36-a24f-b541891e5d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:00:56.538516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:00:56.692010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a157cb0-d562-404c-ab86-6544039c59c1 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:00:56.707691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a157cb0-d562-404c-ab86-6544039c59c1 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:00:56.713756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762872f-8b75-4f09-8095-6be41f93796d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:00:56.714415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762872f-8b75-4f09-8095-6be41f93796d service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:00:56.714701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762872f-8b75-4f09-8095-6be41f93796d service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:00:56.714978 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762872f-8b75-4f09-8095-6be41f93796d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:00:56.723746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762872f-8b75-4f09-8095-6be41f93796d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:00:56.725371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762872f-8b75-4f09-8095-6be41f93796d service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:00:56.734653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 766/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:01:00.974776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63b6c8-e88e-4966-87f5-57624b9690b1 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:00.975333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63b6c8-e88e-4966-87f5-57624b9690b1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:00.975333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63b6c8-e88e-4966-87f5-57624b9690b1 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:00.976710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:01:00.982461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac357db1-e5c4-46b4-b970-797d6213b00d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:00.982955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac357db1-e5c4-46b4-b970-797d6213b00d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:00.983067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac357db1-e5c4-46b4-b970-797d6213b00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:01.001709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ac357db1-e5c4-46b4-b970-797d6213b00d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:01:01.002921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ac357db1-e5c4-46b4-b970-797d6213b00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:01:01.011496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:01:01.048294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63b6c8-e88e-4966-87f5-57624b9690b1 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:01:01.050059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8b63b6c8-e88e-4966-87f5-57624b9690b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:01:01.114752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63b6c8-e88e-4966-87f5-57624b9690b1 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:01:01.118460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e062955-e32c-40d9-aaa4-3547523eec04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:01:01.119114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062955-e32c-40d9-aaa4-3547523eec04 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:01.119397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062955-e32c-40d9-aaa4-3547523eec04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:01.119672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062955-e32c-40d9-aaa4-3547523eec04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:01.129308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e062955-e32c-40d9-aaa4-3547523eec04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:01:01.130752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e062955-e32c-40d9-aaa4-3547523eec04 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:01:01.136566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:01:01.141691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 763/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:01:03.205148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d3137-8422-4743-b7d6-82e06dd17feb None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:03.205732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d3137-8422-4743-b7d6-82e06dd17feb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:03.205732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d3137-8422-4743-b7d6-82e06dd17feb None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:03.207143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:01:03.212396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a705-3186-42ca-86d0-fe9aff73a6ed None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:03.212852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a705-3186-42ca-86d0-fe9aff73a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:03.212958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a705-3186-42ca-86d0-fe9aff73a6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:03.233210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3ae2a705-3186-42ca-86d0-fe9aff73a6ed None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:01:03.234407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3ae2a705-3186-42ca-86d0-fe9aff73a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:01:03.242156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:01:03.278092 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d3137-8422-4743-b7d6-82e06dd17feb None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:01:03.288868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d3137-8422-4743-b7d6-82e06dd17feb None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:01:03.293440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043e311-1dff-4a1a-a1f8-20e34305a814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:01:03.294083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e311-1dff-4a1a-a1f8-20e34305a814 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:03.294369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e311-1dff-4a1a-a1f8-20e34305a814 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:03.294648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e311-1dff-4a1a-a1f8-20e34305a814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:03.304065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043e311-1dff-4a1a-a1f8-20e34305a814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:01:03.306698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043e311-1dff-4a1a-a1f8-20e34305a814 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:01:03.315976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 769/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:01:06.894055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121572b-0041-4aec-838e-8317e53a754b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:06.894656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121572b-0041-4aec-838e-8317e53a754b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:06.894656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121572b-0041-4aec-838e-8317e53a754b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:06.896004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:01:06.902372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccfc6-1a14-40df-924e-6516d2866be8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:06.902845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccfc6-1a14-40df-924e-6516d2866be8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:06.902949 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccfc6-1a14-40df-924e-6516d2866be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:06.920698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8fbccfc6-1a14-40df-924e-6516d2866be8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:01:06.921946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8fbccfc6-1a14-40df-924e-6516d2866be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:01:06.930111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:01:07.241136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121572b-0041-4aec-838e-8317e53a754b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:01:07.243317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2121572b-0041-4aec-838e-8317e53a754b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:01:07.310732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121572b-0041-4aec-838e-8317e53a754b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:01:07.315433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8c6fc-62fb-40be-bd1f-d4e44fea98a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:01:07.316157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c6fc-62fb-40be-bd1f-d4e44fea98a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:07.316466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c6fc-62fb-40be-bd1f-d4e44fea98a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:07.316895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c6fc-62fb-40be-bd1f-d4e44fea98a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:07.327515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8c6fc-62fb-40be-bd1f-d4e44fea98a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:01:07.328941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8c6fc-62fb-40be-bd1f-d4e44fea98a2 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:01:07.338253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 766/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:01:09.418935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f272-efd8-48c3-a84c-67951866a1f1 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:09.419506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f272-efd8-48c3-a84c-67951866a1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:09.419506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f272-efd8-48c3-a84c-67951866a1f1 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:09.420897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:01:09.426246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdb8c4-0b5b-439e-b87d-07971f1acc2d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:09.426693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdb8c4-0b5b-439e-b87d-07971f1acc2d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:09.426802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdb8c4-0b5b-439e-b87d-07971f1acc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:09.445841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-0ccdb8c4-0b5b-439e-b87d-07971f1acc2d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:01:09.447031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0ccdb8c4-0b5b-439e-b87d-07971f1acc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:01:09.454888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:01:09.760174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f272-efd8-48c3-a84c-67951866a1f1 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:01:09.761881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6bf9f272-efd8-48c3-a84c-67951866a1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:01:09.781353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f272-efd8-48c3-a84c-67951866a1f1 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:01:09.785185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2856be9-f31c-458a-92dd-9b369d77cf61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:01:09.785854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2856be9-f31c-458a-92dd-9b369d77cf61 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:01:09.786139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2856be9-f31c-458a-92dd-9b369d77cf61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:01:09.786443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2856be9-f31c-458a-92dd-9b369d77cf61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:01:09.798096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2856be9-f31c-458a-92dd-9b369d77cf61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:01:09.799555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2856be9-f31c-458a-92dd-9b369d77cf61 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:01:09.810560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 772/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:01:19.440511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=32 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:01:19.806139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=70 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:38.486500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a2dd-00fd-4cba-b1a2-1e5af0ea7f5a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.491552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8c1f-a5e1-472b-99b4-3f53523d88c3 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.492586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8c1f-a5e1-472b-99b4-3f53523d88c3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.495420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a2dd-00fd-4cba-b1a2-1e5af0ea7f5a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.496022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8c1f-a5e1-472b-99b4-3f53523d88c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.496617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a2dd-00fd-4cba-b1a2-1e5af0ea7f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.523352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-14b6a2dd-00fd-4cba-b1a2-1e5af0ea7f5a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:38.531080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-14b6a2dd-00fd-4cba-b1a2-1e5af0ea7f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:38.535636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7c9a8c1f-a5e1-472b-99b4-3f53523d88c3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:38.537439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7c9a8c1f-a5e1-472b-99b4-3f53523d88c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:38.545558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 768/1540] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:38.553776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 773/1541] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:38.560802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff283-9993-4490-af6e-d4fea340e060 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.561725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff283-9993-4490-af6e-d4fea340e060 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.562250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff283-9993-4490-af6e-d4fea340e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.568824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd67f4-1dd3-429a-8014-2e2ffef0f249 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.569132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd67f4-1dd3-429a-8014-2e2ffef0f249 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.569688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd67f4-1dd3-429a-8014-2e2ffef0f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.586136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-26bff283-9993-4490-af6e-d4fea340e060 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:38.587896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-26bff283-9993-4490-af6e-d4fea340e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:38.615554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 769/1542] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:38.637847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e699-262b-4b4a-b51d-20ddc3f62f1c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.638220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e699-262b-4b4a-b51d-20ddc3f62f1c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.638505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e699-262b-4b4a-b51d-20ddc3f62f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.678277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fb84e699-262b-4b4a-b51d-20ddc3f62f1c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:38.679685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fb84e699-262b-4b4a-b51d-20ddc3f62f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:38.694881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 770/1543] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:02:38.695790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84e699-262b-4b4a-b51d-20ddc3f62f1c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:38.707945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84e699-262b-4b4a-b51d-20ddc3f62f1c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:38.722838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3bdb6-e8c4-4f27-9d74-3afe86457dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:38.723794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3bdb6-e8c4-4f27-9d74-3afe86457dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.724087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3bdb6-e8c4-4f27-9d74-3afe86457dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.724367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3bdb6-e8c4-4f27-9d74-3afe86457dc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.725140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3bdb6-e8c4-4f27-9d74-3afe86457dc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:38.725453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3bdb6-e8c4-4f27-9d74-3afe86457dc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:38.727090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3bdb6-e8c4-4f27-9d74-3afe86457dc0 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:38.740812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-13dd67f4-1dd3-429a-8014-2e2ffef0f249 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:38.742401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-13dd67f4-1dd3-429a-8014-2e2ffef0f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:38.745814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 771/1544] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:38.747694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc71a-6e9e-42ea-bb05-100987073317 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.748177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc71a-6e9e-42ea-bb05-100987073317 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.748713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc71a-6e9e-42ea-bb05-100987073317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.760275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 774/1545] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:02:38.764285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd67f4-1dd3-429a-8014-2e2ffef0f249 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:38.779093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd67f4-1dd3-429a-8014-2e2ffef0f249 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:38.786173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b95dc71a-6e9e-42ea-bb05-100987073317 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:38.787699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b95dc71a-6e9e-42ea-bb05-100987073317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:38.793793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:38.794872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.795362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.795896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.796841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:38.799049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:38.810537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 772/1546] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:38.811754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc71a-6e9e-42ea-bb05-100987073317 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:38.829003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc71a-6e9e-42ea-bb05-100987073317 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:38.842492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:38.843600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.844150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.844641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.845649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:38.846235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:38.848219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:38.856450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 775/1547] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:38.857359 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:38.870503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 773/1548] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:38.871794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:38.902447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1aec84-10ce-4eea-9f77-41a687979b02 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:38.989420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5141a1c-11d2-4984-9f20-ba6b41959a74 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:38.994993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c573e25-ea37-41c7-bb9b-2ef175664fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:38.995333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:38.996025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.996446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.996644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c573e25-ea37-41c7-bb9b-2ef175664fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:38.996746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.997253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c573e25-ea37-41c7-bb9b-2ef175664fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:38.997755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:38.997983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c573e25-ea37-41c7-bb9b-2ef175664fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:38.998098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:38.998984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c573e25-ea37-41c7-bb9b-2ef175664fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:38.999103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING py.warnings [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:02:38.999103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: warnings.warn(msg) Jun 29 08:02:38.999103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: Jun 29 08:02:38.999992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.001978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c573e25-ea37-41c7-bb9b-2ef175664fa0 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.015169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 776/1549] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:39.016411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.031643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04025b-8022-4f2f-93c8-16be0985066c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.035103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 774/1550] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:38 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.036687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebe903-58e1-4507-9a97-2900f919aeda None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.036996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebe903-58e1-4507-9a97-2900f919aeda None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.037273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcdd1f-1869-4b3b-a365-6b62ce7fb9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.037592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebe903-58e1-4507-9a97-2900f919aeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.037978 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcdd1f-1869-4b3b-a365-6b62ce7fb9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.038379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcdd1f-1869-4b3b-a365-6b62ce7fb9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.038695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcdd1f-1869-4b3b-a365-6b62ce7fb9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.039396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcdd1f-1869-4b3b-a365-6b62ce7fb9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.042069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcdd1f-1869-4b3b-a365-6b62ce7fb9dd admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.057325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e6ebe903-58e1-4507-9a97-2900f919aeda None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.058546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e6ebe903-58e1-4507-9a97-2900f919aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.066680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 775/1551] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.067927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56f0ae-b702-459b-b64d-58c747a15412 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.068212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56f0ae-b702-459b-b64d-58c747a15412 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.068485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56f0ae-b702-459b-b64d-58c747a15412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.074760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 777/1552] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.076088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aec058-d973-4377-9263-7c2c53e3f961 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.076375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aec058-d973-4377-9263-7c2c53e3f961 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.076651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aec058-d973-4377-9263-7c2c53e3f961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.094706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fb56f0ae-b702-459b-b64d-58c747a15412 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.094932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a7aec058-d973-4377-9263-7c2c53e3f961 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.095923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fb56f0ae-b702-459b-b64d-58c747a15412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.096061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a7aec058-d973-4377-9263-7c2c53e3f961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.156935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 776/1553] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.158656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759d6d7-2f07-4b5d-8336-ef8f1c69cfff None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.159037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 778/1554] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.159292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759d6d7-2f07-4b5d-8336-ef8f1c69cfff None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.159852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759d6d7-2f07-4b5d-8336-ef8f1c69cfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.159969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aec058-d973-4377-9263-7c2c53e3f961 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.176327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aec058-d973-4377-9263-7c2c53e3f961 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.182175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2759d6d7-2f07-4b5d-8336-ef8f1c69cfff None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.182524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6da42-14a4-489e-bde1-bb10a43123af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.183334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6da42-14a4-489e-bde1-bb10a43123af admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.183437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2759d6d7-2f07-4b5d-8336-ef8f1c69cfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.183626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6da42-14a4-489e-bde1-bb10a43123af admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.183907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6da42-14a4-489e-bde1-bb10a43123af admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.184619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6da42-14a4-489e-bde1-bb10a43123af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.186670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6da42-14a4-489e-bde1-bb10a43123af admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.192240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 777/1555] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.193104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759d6d7-2f07-4b5d-8336-ef8f1c69cfff None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.203718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759d6d7-2f07-4b5d-8336-ef8f1c69cfff None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.208764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d56ee4-3684-4475-ae7f-d4ffd2953c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.209472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56ee4-3684-4475-ae7f-d4ffd2953c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.209758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56ee4-3684-4475-ae7f-d4ffd2953c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.210034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56ee4-3684-4475-ae7f-d4ffd2953c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.210826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d56ee4-3684-4475-ae7f-d4ffd2953c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.213059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d56ee4-3684-4475-ae7f-d4ffd2953c69 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.222120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 779/1556] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.223913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 778/1557] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:39.225206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba220b2-85b7-46d4-8348-02fb79651497 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.225507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba220b2-85b7-46d4-8348-02fb79651497 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.225782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba220b2-85b7-46d4-8348-02fb79651497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.227101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d98952-a1d9-4928-9788-0b75e9f82ab7 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.227399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d98952-a1d9-4928-9788-0b75e9f82ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.227686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d98952-a1d9-4928-9788-0b75e9f82ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.245471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7ba220b2-85b7-46d4-8348-02fb79651497 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.246619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7ba220b2-85b7-46d4-8348-02fb79651497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.246762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e4d98952-a1d9-4928-9788-0b75e9f82ab7 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.247809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e4d98952-a1d9-4928-9788-0b75e9f82ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.255336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:39.263111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:39.313910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 780/1558] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.315315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d98952-a1d9-4928-9788-0b75e9f82ab7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.326245 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d98952-a1d9-4928-9788-0b75e9f82ab7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.330851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc19040-5d9c-41ee-b2bb-05e07c51a49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.331522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19040-5d9c-41ee-b2bb-05e07c51a49e admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.331810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19040-5d9c-41ee-b2bb-05e07c51a49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.332107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19040-5d9c-41ee-b2bb-05e07c51a49e admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.332803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc19040-5d9c-41ee-b2bb-05e07c51a49e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.337277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 779/1559] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.337739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING py.warnings [None req-4dc19040-5d9c-41ee-b2bb-05e07c51a49e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:02:39.337739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: warnings.warn(msg) Jun 29 08:02:39.337739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: Jun 29 08:02:39.338180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a96bf8-751c-41d6-9c55-dc0bcc3b29b7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.338327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc19040-5d9c-41ee-b2bb-05e07c51a49e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.338463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a96bf8-751c-41d6-9c55-dc0bcc3b29b7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.338463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a96bf8-751c-41d6-9c55-dc0bcc3b29b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.355464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f4a96bf8-751c-41d6-9c55-dc0bcc3b29b7 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.357147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f4a96bf8-751c-41d6-9c55-dc0bcc3b29b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.363770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 781/1560] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.363770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ee5e4-f362-4526-b53a-b4a9da963958 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.363770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ee5e4-f362-4526-b53a-b4a9da963958 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.363770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ee5e4-f362-4526-b53a-b4a9da963958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.365071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 780/1561] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.366487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e7576-47dd-44e5-b5b3-eafb9290f384 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.366867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e7576-47dd-44e5-b5b3-eafb9290f384 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.367031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e7576-47dd-44e5-b5b3-eafb9290f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.382813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-594ee5e4-f362-4526-b53a-b4a9da963958 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.384664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-594ee5e4-f362-4526-b53a-b4a9da963958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.386787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-188e7576-47dd-44e5-b5b3-eafb9290f384 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.387976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-188e7576-47dd-44e5-b5b3-eafb9290f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.393132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 782/1562] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.395985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 781/1563] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.396987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e7576-47dd-44e5-b5b3-eafb9290f384 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.397133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ee5e4-f362-4526-b53a-b4a9da963958 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.406973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ee5e4-f362-4526-b53a-b4a9da963958 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.407492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e7576-47dd-44e5-b5b3-eafb9290f384 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.411977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.412365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b45763-abcc-4b48-87b9-2082e6633c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.412646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.412929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.413075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45763-abcc-4b48-87b9-2082e6633c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.413252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.413362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45763-abcc-4b48-87b9-2082e6633c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.413629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45763-abcc-4b48-87b9-2082e6633c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.413927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.414352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b45763-abcc-4b48-87b9-2082e6633c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.416501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b45763-abcc-4b48-87b9-2082e6633c11 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.416741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.427708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 782/1564] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:39.429251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d559ab-886f-43aa-b1c5-6ffc8a1ae35e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.429647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d559ab-886f-43aa-b1c5-6ffc8a1ae35e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.429982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d559ab-886f-43aa-b1c5-6ffc8a1ae35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.442301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 783/1565] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.443221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.453005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e1d559ab-886f-43aa-b1c5-6ffc8a1ae35e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.453344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daee28-b573-4dcf-9f12-ef4576f7da8d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.454263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e1d559ab-886f-43aa-b1c5-6ffc8a1ae35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.457970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.458636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.458920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.459196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.459921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:39.460229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.461879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.465464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 783/1566] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.465464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d559ab-886f-43aa-b1c5-6ffc8a1ae35e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.473517 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d559ab-886f-43aa-b1c5-6ffc8a1ae35e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.477314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 784/1567] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:39.478181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.478363 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.479000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.479281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.479563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.481018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.484084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.490174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b7cf7-2c42-4cf6-b73b-5ec61bc896d2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.497436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 784/1568] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:39.497436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.498885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.499563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.499847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.500123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.500960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:39.501180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.502810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.508008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000283c-afd6-41f5-b98c-987edb052498 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.513186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.513841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.514123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.514398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.515115 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:39.515418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.515910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 785/1569] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:39.516593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.517396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.526874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0759305-d2b5-4de5-99a0-1c13dbfb6839 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.537601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 785/1570] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:39.537806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.537949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.538053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.538053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.538053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.538053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:39.538053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.538053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.544378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc190-97e4-4fa3-8216-dda620064966 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.549031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.549915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.550200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.557540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 786/1571] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:39.557743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] PATH_INFO: `/v3/projects/ab693125979f4bb7b7b3c94061610fe2/users/1f4af4eedee24897be2e702ef51873b3/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.557852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.561804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b91d-5adb-4b3d-a42a-29c9e348b396 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.566654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.567322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.567609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.567890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.568625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.570266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.574843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab693125979f4bb7b7b3c94061610fe2, user_id=1f4af4eedee24897be2e702ef51873b3, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.575887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:02:39.575887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 08:02:39.575887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 08:02:39.577841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.581285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 787/1572] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:39.582123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.593287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc411c-8a9d-480b-b102-ec96d255d5ac admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.594616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 786/1573] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:39 2022] PUT /identity/v3/projects/ab693125979f4bb7b7b3c94061610fe2/users/1f4af4eedee24897be2e702ef51873b3/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:39.598256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.652806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.653516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.653805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.654087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.654837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:39.655136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.656749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.661389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412624-2e04-4178-900b-d14373a0fc70 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.667813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.668487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.668777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.669095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.669379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 788/1574] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:39.669938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.670205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.671041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:02:39.671041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 08:02:39.671041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 08:02:39.672277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.686202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd65160-8293-41b3-95b0-f7cc8bff92ba admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.691240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.691930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.692213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.692588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.693486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.695334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.704240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 787/1575] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.705103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.711118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 789/1576] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:39.712809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.724233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b38e3a-952d-4996-b187-5763dfa87a5b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.727996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:39.730786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:39.775779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c930-0103-46ee-aa53-cc34c4d3a1e8 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.780468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.781153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.781438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.781739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.782431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.784291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.784430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.784963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.785280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.785585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.786278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.788188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.815654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 788/1577] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.816317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.824744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 790/1578] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.824744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.886969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88717d-fe27-4b45-8d76-9b54914b7aa8 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.887678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320790b-fbe8-4780-9b3d-cf303a9d0ca0 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.891654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.892320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.892433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8802643-84dd-4393-89a8-2141c40468c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:39.892601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.892881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.893013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8802643-84dd-4393-89a8-2141c40468c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.893299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8802643-84dd-4393-89a8-2141c40468c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.893580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8802643-84dd-4393-89a8-2141c40468c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.893686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.894316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8802643-84dd-4393-89a8-2141c40468c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:39.895539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.895911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8802643-84dd-4393-89a8-2141c40468c9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:39.907695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 791/1579] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:39.909111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b44acf-51c8-4370-a101-265a11ee9202 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:39.909530 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b44acf-51c8-4370-a101-265a11ee9202 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:39.909816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b44acf-51c8-4370-a101-265a11ee9202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:39.928059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 789/1580] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:39.928059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.929615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c8b44acf-51c8-4370-a101-265a11ee9202 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:39.931368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c8b44acf-51c8-4370-a101-265a11ee9202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:39.991875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 792/1581] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:39.992784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b44acf-51c8-4370-a101-265a11ee9202 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:39.994226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbccf8-0970-4e39-a4dc-35c8e558eabd admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:39.999829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.000540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.000829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.001155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.001895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.003835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.005284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b44acf-51c8-4370-a101-265a11ee9202 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.010140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.010817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.011129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.011406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.012131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.014094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.036206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 790/1582] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.036206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.036470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 793/1583] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:39 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.036470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.046916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7c88c-6ba8-4dfe-ba8e-c914497546e0 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.055260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:40.057901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:40.105343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6778cb-45a8-4a6e-b307-0bca9a6fb726 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.109471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.109994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.110204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.110438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.110715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.110885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.111175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.111407 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.111514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] PATH_INFO: `/v3/projects/064750b913ab4e74bd7cdb820afec6f8/users/fe6e8d767dc7489ba390ce3a229fe35a/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.113396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.127922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=064750b913ab4e74bd7cdb820afec6f8, user_id=fe6e8d767dc7489ba390ce3a229fe35a, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.130848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.135777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 794/1584] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.136762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.146958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 791/1585] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:40 2022] PUT /identity/v3/projects/064750b913ab4e74bd7cdb820afec6f8/users/fe6e8d767dc7489ba390ce3a229fe35a/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:40.147847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b48dbb-8afb-480d-95f5-eeb0517e0000 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.149920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.212316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.213305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.213597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.213884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.214590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.216604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.221090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec8323-f39e-438c-94af-7b3277b16a3c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.225684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.226650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.226969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.227280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] PATH_INFO: `/v3/projects/61cd7366997e4508a1a553eb109695af/users/0b66efcbfeb74648a3a52fb6df10195a/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.239413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 795/1586] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.240697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.244469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61cd7366997e4508a1a553eb109695af, user_id=0b66efcbfeb74648a3a52fb6df10195a, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.247194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.251713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fb215-d285-43b4-bcda-41482f947e10 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.256368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.257061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.257349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.257626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.258334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.261000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.262726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 792/1587] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:40 2022] PUT /identity/v3/projects/61cd7366997e4508a1a553eb109695af/users/0b66efcbfeb74648a3a52fb6df10195a/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:40.263638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.291849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 796/1588] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/users => generated 295 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.291849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.299437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e6f7c-4665-4c59-9fc8-88692ba08c7b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.330521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e541ce5-bb0e-4f34-9c30-773efeb6e11b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.334155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a783b-4389-457d-9d4d-42becaea3688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.334808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a783b-4389-457d-9d4d-42becaea3688 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.335205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a783b-4389-457d-9d4d-42becaea3688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.335487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a783b-4389-457d-9d4d-42becaea3688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.356875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.357583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.357871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.358453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.359152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.361098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.371957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:40.382880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 797/1589] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.383801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.390225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:40.397559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96469d-6edf-4103-8994-5a7f6d76e6a6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.403187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.403909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.404194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.404947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.405233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.406864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a783b-4389-457d-9d4d-42becaea3688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.406991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.408302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a783b-4389-457d-9d4d-42becaea3688 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.417048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 798/1590] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.417882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.418460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 793/1591] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:40.419753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0314341-f77b-4dbb-b9f3-4276dec32264 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.420037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0314341-f77b-4dbb-b9f3-4276dec32264 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.420447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0314341-f77b-4dbb-b9f3-4276dec32264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.427925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfaf83f-87e7-4c41-882a-4a82962bee56 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.433020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.433699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.434016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.434252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.435158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.436818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.448251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 799/1592] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.449137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.450459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b0314341-f77b-4dbb-b9f3-4276dec32264 None None] MFA Rules not processed for user `fe6e8d767dc7489ba390ce3a229fe35a`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:40.452009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b0314341-f77b-4dbb-b9f3-4276dec32264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:40.459651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b657-460d-4d4d-875b-93d6b20e0e55 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.465605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32944398-d733-42b9-9667-4860dedc6b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.469647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32944398-d733-42b9-9667-4860dedc6b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.470297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32944398-d733-42b9-9667-4860dedc6b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.471416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32944398-d733-42b9-9667-4860dedc6b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.472630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32944398-d733-42b9-9667-4860dedc6b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.475114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32944398-d733-42b9-9667-4860dedc6b3c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.491225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 800/1593] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.492811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7295a-ba4e-4ebb-9cc8-6f564707d8bd None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.493009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7295a-ba4e-4ebb-9cc8-6f564707d8bd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.494098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7295a-ba4e-4ebb-9cc8-6f564707d8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.522319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-01d7295a-ba4e-4ebb-9cc8-6f564707d8bd None None] MFA Rules not processed for user `0b66efcbfeb74648a3a52fb6df10195a`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:40.523536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-01d7295a-ba4e-4ebb-9cc8-6f564707d8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:40.534184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 794/1594] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:40.536198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0314341-f77b-4dbb-b9f3-4276dec32264 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.550128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0314341-f77b-4dbb-b9f3-4276dec32264 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.555599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.556576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.556929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.557350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.558166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.559914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.571979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 795/1595] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.573420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.584318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd41609-2921-46d8-9a1e-1cddad95603c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.591221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.591994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.592272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.592593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.593443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.595174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.607804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 796/1596] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.608842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.623111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 801/1597] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:40.626595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7295a-ba4e-4ebb-9cc8-6f564707d8bd None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.632932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe3ea-f7a2-4c54-81e9-da39f82ccfb8 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.646462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.647259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.647508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.647793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.648540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.650637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7295a-ba4e-4ebb-9cc8-6f564707d8bd None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.650919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.656331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.657431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.657857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.658284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.659297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.661272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.662881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 797/1598] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.663491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.675557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ede3b1-3178-4ec2-9266-07d748e502b0 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.681027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 802/1599] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.681027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.684683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.685397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.685701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.685977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.686717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.688377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.692925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e11d4-25a4-4183-98ba-59beadcb1252 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.699761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.699985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 798/1600] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.700205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.700372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.700497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.700775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.701505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.703768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.711403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9577ce-cf25-4cc5-8f12-4b8d2369e6bc admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.718944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.719619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.719916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.720179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.720922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.722612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.735689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 799/1601] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.736681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.741045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 803/1602] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.742276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.750183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63fde6-4c88-4cea-be47-0f665c2fb678 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.818996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.819794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.820029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.820569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.821362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.824112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.829577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa99880-738e-4565-b64d-a478180e769d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.834830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.835510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.835794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.836105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.837832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.839775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.840011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 800/1603] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:40.840893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.846982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:40.856099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c2b61-a2fe-45b7-bcd8-bd9475a8791a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:40.865130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:40.866912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:40.867088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:40.868136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] PATH_INFO: `/v3/projects/f46c6cdc3217456dba6eda8ff23e9f3a/users/0800a22846504b0d9c49e1520a69e45b/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:40.897971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 804/1604] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:40 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:40.899151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:40.901461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46c6cdc3217456dba6eda8ff23e9f3a, user_id=0800a22846504b0d9c49e1520a69e45b, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:40.908013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:40.922795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:40.941210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 801/1605] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:40 2022] PUT /identity/v3/projects/f46c6cdc3217456dba6eda8ff23e9f3a/users/0800a22846504b0d9c49e1520a69e45b/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:40.942303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.006555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8476448-a0a5-436d-a070-c158030343ee admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.020995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0d42-e251-40c2-a7af-25523345188c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.026714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.027632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.027918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.028214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] PATH_INFO: `/v3/projects/b0f963a60fc147fda6d8bb638607247b/users/043c4bb69b7d480db456c1212a14112b/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.044671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f963a60fc147fda6d8bb638607247b, user_id=043c4bb69b7d480db456c1212a14112b, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.047408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.064508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 802/1606] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:40 2022] PUT /identity/v3/projects/b0f963a60fc147fda6d8bb638607247b/users/043c4bb69b7d480db456c1212a14112b/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:41.065715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.089561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.090765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.090765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.090993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] PATH_INFO: `/v3/projects/50485aa2c4f9419ba622b4a278671ec7/users/2283d984870a47f3bcf913aca5f98670/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.108442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50485aa2c4f9419ba622b4a278671ec7, user_id=2283d984870a47f3bcf913aca5f98670, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.111151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.131504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 805/1607] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:40 2022] PUT /identity/v3/projects/50485aa2c4f9419ba622b4a278671ec7/users/2283d984870a47f3bcf913aca5f98670/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:41.132790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.158343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748254-34f8-428d-944e-693b664761ff admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.221854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf50d2-803e-470b-a5d1-6bfa4ac23364 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.229317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.230184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.230634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.230973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.232128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.235022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.250627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.251724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.252005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.252284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] PATH_INFO: `/v3/projects/4cd09897be9c4642b91dfb53a536e7d1/users/c9065575ed314f6f8f3915bf596911e5/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.275230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:41.277029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 806/1608] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:41.277029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.281102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd09897be9c4642b91dfb53a536e7d1, user_id=c9065575ed314f6f8f3915bf596911e5, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.287573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.288163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67daf9-63a5-404a-9c00-924f11de5527 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.296835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a25168-526b-44f7-9a0b-fe6a58e552a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.298388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a25168-526b-44f7-9a0b-fe6a58e552a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.298690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a25168-526b-44f7-9a0b-fe6a58e552a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.298980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a25168-526b-44f7-9a0b-fe6a58e552a3 admin admin] PATH_INFO: `/v3/projects/5e0c37d941434569b3dbd07e22125210/users/c70c44993ab240b1a5231df4c9d550c9/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.311445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 803/1609] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:41 2022] PUT /identity/v3/projects/4cd09897be9c4642b91dfb53a536e7d1/users/c9065575ed314f6f8f3915bf596911e5/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:41.313899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.318402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a25168-526b-44f7-9a0b-fe6a58e552a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0c37d941434569b3dbd07e22125210, user_id=c70c44993ab240b1a5231df4c9d550c9, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.322552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a25168-526b-44f7-9a0b-fe6a58e552a3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.343696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 807/1610] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:41 2022] PUT /identity/v3/projects/5e0c37d941434569b3dbd07e22125210/users/c70c44993ab240b1a5231df4c9d550c9/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:41.345261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca99d-9e8e-489e-9ced-714ee71c99d1 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.345559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca99d-9e8e-489e-9ced-714ee71c99d1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.345845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca99d-9e8e-489e-9ced-714ee71c99d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.362646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:41.370456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1c4ca99d-9e8e-489e-9ced-714ee71c99d1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:41.372274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1c4ca99d-9e8e-489e-9ced-714ee71c99d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:41.407612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264ac46-822b-4ada-b05c-b336a1ce0651 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.451685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 808/1611] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:41.461855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63960a5-25a2-4996-997b-e97c19584b4f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.463011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63960a5-25a2-4996-997b-e97c19584b4f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.463305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63960a5-25a2-4996-997b-e97c19584b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.504217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f63960a5-25a2-4996-997b-e97c19584b4f None None] MFA Rules not processed for user `043c4bb69b7d480db456c1212a14112b`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:41.505730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f63960a5-25a2-4996-997b-e97c19584b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:41.512491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a682e0-f642-4d6d-b2b8-d1fdd36f9916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.513394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a682e0-f642-4d6d-b2b8-d1fdd36f9916 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.513753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a682e0-f642-4d6d-b2b8-d1fdd36f9916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.514099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a682e0-f642-4d6d-b2b8-d1fdd36f9916 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.514911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a682e0-f642-4d6d-b2b8-d1fdd36f9916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.517720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a682e0-f642-4d6d-b2b8-d1fdd36f9916 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.551667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 804/1612] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:41.553384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.553844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.554199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.583336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 809/1613] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:41.584331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63960a5-25a2-4996-997b-e97c19584b4f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.598539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63960a5-25a2-4996-997b-e97c19584b4f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.599170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] MFA Rules not processed for user `2283d984870a47f3bcf913aca5f98670`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:41.600744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:41.605144 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecbde9-318f-4f99-a9cd-79286a605206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.606140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecbde9-318f-4f99-a9cd-79286a605206 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.606442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecbde9-318f-4f99-a9cd-79286a605206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.606817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecbde9-318f-4f99-a9cd-79286a605206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.607796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecbde9-318f-4f99-a9cd-79286a605206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.610310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecbde9-318f-4f99-a9cd-79286a605206 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.624596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 810/1614] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:41.626197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423b6cc-bc87-499e-8b54-8824e7749e90 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.626491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423b6cc-bc87-499e-8b54-8824e7749e90 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.626772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423b6cc-bc87-499e-8b54-8824e7749e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.650313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8423b6cc-bc87-499e-8b54-8824e7749e90 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:41.651815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8423b6cc-bc87-499e-8b54-8824e7749e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:41.664534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 811/1615] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:41.667561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9abb-2573-4a60-a150-36373cf3e9bd None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.667836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9abb-2573-4a60-a150-36373cf3e9bd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.668117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9abb-2573-4a60-a150-36373cf3e9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.698354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 805/1616] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:41.704534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.708438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:41.729410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-566f9abb-2573-4a60-a150-36373cf3e9bd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:41.730983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-566f9abb-2573-4a60-a150-36373cf3e9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:41.746169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 812/1617] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:41.746169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f9abb-2573-4a60-a150-36373cf3e9bd None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.753245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:41.757550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f9abb-2573-4a60-a150-36373cf3e9bd None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.764528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.765515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.765955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.766634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.768108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.771809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.796819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 813/1618] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:41.797507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53609818-6ae6-43ad-bc08-2c7e0d8bb24e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.797807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.803389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.804057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.804348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.804633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.809361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ee458-2dcc-4f56-aa2d-2b426db491c2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.814613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.815917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.816217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.816500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:41.817156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.817297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:41.818620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.819737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:41.842009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 806/1619] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:41.845324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.865777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 814/1620] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:41 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:41.872813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:41.880402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a4a46-77e6-4c5d-bd5a-19ec5e03612b service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.950510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:41.969655 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016e6cf-7720-44a8-9e53-3bea4c48f8c3 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:41.978629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:41.979370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:41.981411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:41.981744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.003984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.004900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.005506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.005506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] PATH_INFO: `/v3/projects/50c7c44edb3c461e952d2a3f1b5da059/users/8e5996e3bbcf40ca9e51b53efa90cff2/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.063197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c7c44edb3c461e952d2a3f1b5da059, user_id=8e5996e3bbcf40ca9e51b53efa90cff2, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.066481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.090502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 807/1621] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:41 2022] PUT /identity/v3/projects/50c7c44edb3c461e952d2a3f1b5da059/users/8e5996e3bbcf40ca9e51b53efa90cff2/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:42.091414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.125664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.127338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.140258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 815/1622] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:42.141379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.154517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503b294-b16b-4c4d-83b2-4aad69204bae service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.209446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1d083-7e39-405b-bdc0-7d989f0202db admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.216126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.216636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.216928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.217251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.217976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.220511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.265718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 808/1623] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:42 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:42.267290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.364315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.364315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.364315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.364315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.366374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:42.367215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.367215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.375627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 816/1624] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:42.375627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.397879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d686baa-90a7-41f5-bf0c-11a1bd340091 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.465218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb821aa-5d9f-46a8-8368-f438443bf5d0 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.482893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.484716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.485048 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.485342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] PATH_INFO: `/v3/projects/cb1f414294364c06be631ba97eead63d/users/8573e7c4d2d4404b9337cf16358a40dd/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.493721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.494501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.494845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.495137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.496412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.499520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.513250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1f414294364c06be631ba97eead63d, user_id=8573e7c4d2d4404b9337cf16358a40dd, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.517739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.534197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 817/1625] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:42 2022] POST /identity/v3/users => generated 300 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:42.534934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.540644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 809/1626] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:42 2022] PUT /identity/v3/projects/cb1f414294364c06be631ba97eead63d/users/8573e7c4d2d4404b9337cf16358a40dd/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:42.542840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.548872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8dff1-23d6-442c-ac58-fab678885ef2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.579310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:42.724574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18861a-249c-4faf-b269-38e3018cf1b7 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.737179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.739389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.741100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.742704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.743758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.748794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.751413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.757382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.758189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.760411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.762183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.767826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.791561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 818/1627] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:42.794666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.832709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 810/1628] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:42 2022] POST /identity/v3/users => generated 289 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:42.834957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.835358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dcaec-11e9-4a17-916d-3b21053b16fc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.861706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.862675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.862965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.863915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.864738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.869301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.892781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 819/1629] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:42.893832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.894895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f6673-d699-4ccc-a97b-98314af6eb25 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.912321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee9260-e8e3-400e-852d-afc978da2547 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.912593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.913951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.914615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.915563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.916733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.919764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.926620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:42.928232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:42.928232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:42.928232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] PATH_INFO: `/v3/projects/50c7c44edb3c461e952d2a3f1b5da059/users/8e5996e3bbcf40ca9e51b53efa90cff2/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:42.935414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c7c44edb3c461e952d2a3f1b5da059, user_id=8e5996e3bbcf40ca9e51b53efa90cff2, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:42.947419 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:42.966909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 811/1630] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:42.967069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:42.990914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913b48-0278-45b0-974f-17b7cefe0a81 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:42.992894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 820/1631] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:42 2022] PUT /identity/v3/projects/50c7c44edb3c461e952d2a3f1b5da059/users/8e5996e3bbcf40ca9e51b53efa90cff2/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:43.006507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:43.176326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:43.176949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.177264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.177553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.178430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:43.181250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:43.216855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 812/1632] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:43.216855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:43.235396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14844183-b36e-419d-bacb-03a28121dc46 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:43.246470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf0d84-5966-4189-b42f-2eb3acfe1cbf admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:43.249041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5a813-5c6d-4652-b3d1-245e6c8cc258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:43.250304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5a813-5c6d-4652-b3d1-245e6c8cc258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.250626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5a813-5c6d-4652-b3d1-245e6c8cc258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.250996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5a813-5c6d-4652-b3d1-245e6c8cc258 admin admin] PATH_INFO: `/v3/projects/cb1f414294364c06be631ba97eead63d/users/8573e7c4d2d4404b9337cf16358a40dd/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.254943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:43.255566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5a813-5c6d-4652-b3d1-245e6c8cc258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1f414294364c06be631ba97eead63d, user_id=8573e7c4d2d4404b9337cf16358a40dd, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:43.255922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.256213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.256493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] PATH_INFO: `/v3/projects/fa03c3fdcb7343e695f5a2be2da02465/users/d6e34e42411d4352a39dc7b14f842ac4/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.262455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5a813-5c6d-4652-b3d1-245e6c8cc258 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:43.283080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa03c3fdcb7343e695f5a2be2da02465, user_id=d6e34e42411d4352a39dc7b14f842ac4, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:43.287608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:43.292918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 813/1633] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:43 2022] PUT /identity/v3/projects/cb1f414294364c06be631ba97eead63d/users/8573e7c4d2d4404b9337cf16358a40dd/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:43.294819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72415cae-68fc-4dd0-ba4c-fa3af5c20854 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.295114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72415cae-68fc-4dd0-ba4c-fa3af5c20854 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.295397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72415cae-68fc-4dd0-ba4c-fa3af5c20854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.344074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 821/1634] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:42 2022] PUT /identity/v3/projects/fa03c3fdcb7343e695f5a2be2da02465/users/d6e34e42411d4352a39dc7b14f842ac4/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:43.344645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:43.348690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:43.356419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-72415cae-68fc-4dd0-ba4c-fa3af5c20854 None None] MFA Rules not processed for user `8e5996e3bbcf40ca9e51b53efa90cff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:43.358218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-72415cae-68fc-4dd0-ba4c-fa3af5c20854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:43.525807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 814/1635] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:43.527948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93e2a8-7040-4dab-8bc4-c2251d0029b8 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:43.532349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de9e1-8f7c-4491-bfc2-6838f088e55f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.533171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de9e1-8f7c-4491-bfc2-6838f088e55f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.535518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de9e1-8f7c-4491-bfc2-6838f088e55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.542571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:43.544407 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.544707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.547079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] PATH_INFO: `/v3/projects/ed834c17f32644d6bb89b55645dd97a4/users/019c15c3a392460697fd7afff02f07d0/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.582260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed834c17f32644d6bb89b55645dd97a4, user_id=019c15c3a392460697fd7afff02f07d0, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:43.585152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:43.590525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c69de9e1-8f7c-4491-bfc2-6838f088e55f None None] MFA Rules not processed for user `8573e7c4d2d4404b9337cf16358a40dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:43.591485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c69de9e1-8f7c-4491-bfc2-6838f088e55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:43.610569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:43.627926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 822/1636] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:43 2022] PUT /identity/v3/projects/ed834c17f32644d6bb89b55645dd97a4/users/019c15c3a392460697fd7afff02f07d0/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:43.627926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:43.731563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 815/1637] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:43.734026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69de9e1-8f7c-4491-bfc2-6838f088e55f None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:43.749241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79cd29-1ed1-469a-ad91-4920951c3de5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:43.756207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:43.757250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.758262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.758633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.760558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:43.763750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:43.780101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 823/1638] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:43.781427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:43.796505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46c5bb-91ad-4751-a8d8-c09d8ba6691f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:43.805354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:43.807329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.807702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.808184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:43.809694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:43.815619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:43.816210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69de9e1-8f7c-4491-bfc2-6838f088e55f None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:43.844347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 824/1639] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:43.848284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:43.924821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:43.964334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:43.969094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:43.969568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:43.969864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.027945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f6e88-666b-468f-ae02-05ee202a7e17 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:44.041899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:44.043442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:44.043736 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:44.045159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.104731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:44.108549 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:44.150371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 816/1640] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:44.153689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:44.197659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7038c8-4b13-4d54-a281-179168179de3 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:44.213788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784751f-081d-4565-9c48-5cd87085106a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:44.214839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784751f-081d-4565-9c48-5cd87085106a admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:44.215851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784751f-081d-4565-9c48-5cd87085106a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:44.216255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784751f-081d-4565-9c48-5cd87085106a admin admin] PATH_INFO: `/v3/projects/fa03c3fdcb7343e695f5a2be2da02465/users/d6e34e42411d4352a39dc7b14f842ac4/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.221166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784751f-081d-4565-9c48-5cd87085106a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa03c3fdcb7343e695f5a2be2da02465, user_id=d6e34e42411d4352a39dc7b14f842ac4, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:44.223123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:44.225297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:44.230783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784751f-081d-4565-9c48-5cd87085106a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:44.251581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 825/1641] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:44.252437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:44.265352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 817/1642] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:44 2022] PUT /identity/v3/projects/fa03c3fdcb7343e695f5a2be2da02465/users/d6e34e42411d4352a39dc7b14f842ac4/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:44.270098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171716b-6718-4e99-b441-d69fe4ef573c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:44.271555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171716b-6718-4e99-b441-d69fe4ef573c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:44.272027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171716b-6718-4e99-b441-d69fe4ef573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.281039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c2e9b-543d-4baf-8ccf-121d5d9eda7f service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:44.353095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c171716b-6718-4e99-b441-d69fe4ef573c None None] MFA Rules not processed for user `d6e34e42411d4352a39dc7b14f842ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:44.355353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c171716b-6718-4e99-b441-d69fe4ef573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:44.370647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:44.433305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:44.436490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:44.438910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:44.439231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] PATH_INFO: `/v3/projects/ed834c17f32644d6bb89b55645dd97a4/users/019c15c3a392460697fd7afff02f07d0/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.446355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed834c17f32644d6bb89b55645dd97a4, user_id=019c15c3a392460697fd7afff02f07d0, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:44.459803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:44.501254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 826/1643] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:44 2022] PUT /identity/v3/projects/ed834c17f32644d6bb89b55645dd97a4/users/019c15c3a392460697fd7afff02f07d0/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:44.502268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:44.581630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 818/1644] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:44.583334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b89358-29b0-482b-852d-f30aa3d032f2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:44.585758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b89358-29b0-482b-852d-f30aa3d032f2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:44.587953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b89358-29b0-482b-852d-f30aa3d032f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.642980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9e198-748d-4de4-b683-2cc2a5516c1b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:44.647964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f6b89358-29b0-482b-852d-f30aa3d032f2 None None] MFA Rules not processed for user `019c15c3a392460697fd7afff02f07d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:44.649882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f6b89358-29b0-482b-852d-f30aa3d032f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:44.651935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:44.652630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:44.652918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:44.653229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.729025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:44.761737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 819/1645] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:44.763355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b89358-29b0-482b-852d-f30aa3d032f2 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:44.778224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b89358-29b0-482b-852d-f30aa3d032f2 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:44.781254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:44.783014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:44.783288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:44.784049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:44.784834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:44.785442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:44.806267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 827/1646] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:44.869149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:44.872199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:44.903319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 820/1647] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:44.956558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:45.021122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:45.041612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec8ac3-c95f-43f4-bc58-c5eb670d5166 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:45.053291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a603f-0b38-4407-8299-9d02dc678c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:45.057362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a603f-0b38-4407-8299-9d02dc678c5e service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:45.057719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a603f-0b38-4407-8299-9d02dc678c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:45.061063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a603f-0b38-4407-8299-9d02dc678c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:45.091840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a603f-0b38-4407-8299-9d02dc678c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:45.095639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a603f-0b38-4407-8299-9d02dc678c5e service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:45.121891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 821/1648] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:02:45.122227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8dec7-ddd2-4a94-ae9d-7c0a7ee2fce6 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:45.133378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:45.135926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:45.137336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:45.137748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] PATH_INFO: `/v3/users/043c4bb69b7d480db456c1212a14112b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:45.142148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=043c4bb69b7d480db456c1212a14112b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:45.146073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:45.217112 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:45.273774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043c4bb69b7d480db456c1212a14112b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:45.275333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043c4bb69b7d480db456c1212a14112b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:45.303009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043c4bb69b7d480db456c1212a14112b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:45.304190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043c4bb69b7d480db456c1212a14112b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:45.340165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043c4bb69b7d480db456c1212a14112b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:45.393651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 828/1649] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:02:44 2022] DELETE /identity/v3/users/043c4bb69b7d480db456c1212a14112b => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:45.398613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:45.398922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:45.399257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:45.405097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:45.406671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:45.464386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:45.479404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:45.627532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb920ae-748a-4259-bc11-40c7d1fbeae9 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:45.642390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:45.643447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:45.643915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:45.645147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:45.650168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:45.732464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 822/1650] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:45.785807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:45.787211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:45.804787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:45.813686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:45.857483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 829/1651] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:45.985942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:45.987856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:45.998410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950b31b-8f8d-45c5-aefc-7fa00c8f61e8 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:46.004668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9afa2-e1c9-4089-b3f9-2b9caab32373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:46.006345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9afa2-e1c9-4089-b3f9-2b9caab32373 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:46.006345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9afa2-e1c9-4089-b3f9-2b9caab32373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:46.006581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9afa2-e1c9-4089-b3f9-2b9caab32373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:46.015057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab720-ef89-4027-827e-72c7e841c4fb service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:46.030357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:46.032597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:46.033957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:46.034249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] PATH_INFO: `/v3/users/fe6e8d767dc7489ba390ce3a229fe35a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:46.037504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe6e8d767dc7489ba390ce3a229fe35a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:46.044381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:46.075769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9afa2-e1c9-4089-b3f9-2b9caab32373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:46.083075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9afa2-e1c9-4089-b3f9-2b9caab32373 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:46.114567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 823/1652] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:02:46.213476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe6e8d767dc7489ba390ce3a229fe35a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:46.216048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe6e8d767dc7489ba390ce3a229fe35a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:46.232446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:46.237888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe6e8d767dc7489ba390ce3a229fe35a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:46.238894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe6e8d767dc7489ba390ce3a229fe35a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:46.270885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe6e8d767dc7489ba390ce3a229fe35a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:46.298982 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 830/1653] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:02:45 2022] DELETE /identity/v3/users/fe6e8d767dc7489ba390ce3a229fe35a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:46.303678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:46.304126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:46.304499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:46.337573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:46.339680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:46.346875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:46.459873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 824/1654] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:46.511480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:46.513200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:46.604942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad3ab-82bc-4555-a250-ba0ad2559897 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:46.609489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:46.610585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:46.611054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:46.611539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:46.627330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:46.629273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:46.646443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 831/1655] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:47.804267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:47.806494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:47.834925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1210f2-b517-43ce-beff-d54b95063f9b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:47.847526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:47.849242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:47.849671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:47.849969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] PATH_INFO: `/v3/projects/b0f963a60fc147fda6d8bb638607247b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:47.854992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f963a60fc147fda6d8bb638607247b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:47.866120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:47.969614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f963a60fc147fda6d8bb638607247b', 'request_id': 'req-aadad1d9-3c08-42f6-b2be-e42767aeabd3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.000548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f963a60fc147fda6d8bb638607247b', 'request_id': 'req-aadad1d9-3c08-42f6-b2be-e42767aeabd3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.035631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f963a60fc147fda6d8bb638607247b', 'request_id': 'req-aadad1d9-3c08-42f6-b2be-e42767aeabd3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.065351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] The token cache is being invalidate because project b0f963a60fc147fda6d8bb638607247b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:02:48.066887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.068629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.081588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 825/1656] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:02:47 2022] DELETE /identity/v3/projects/b0f963a60fc147fda6d8bb638607247b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:02:48.085793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:48.102617 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f335c-45f3-4dbe-9887-9a93e7b44988 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:48.366131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:48.370063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:48.370934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:48.370934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] PATH_INFO: `/v3/users/d6e34e42411d4352a39dc7b14f842ac4` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:48.376282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e34e42411d4352a39dc7b14f842ac4)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:48.378316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:48.559158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e34e42411d4352a39dc7b14f842ac4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.562939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e34e42411d4352a39dc7b14f842ac4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.580617 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e34e42411d4352a39dc7b14f842ac4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.581549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e34e42411d4352a39dc7b14f842ac4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.597645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e34e42411d4352a39dc7b14f842ac4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:48.600237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:48.651462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 832/1657] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:02:48 2022] DELETE /identity/v3/users/d6e34e42411d4352a39dc7b14f842ac4 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:48.967407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:48.972673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:48.992723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadad1d9-3c08-42f6-b2be-e42767aeabd3 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:49.224293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:49.225768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:49.226081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:49.226383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] PATH_INFO: `/v3/projects/064750b913ab4e74bd7cdb820afec6f8` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:49.228375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=064750b913ab4e74bd7cdb820afec6f8)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:49.235023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:49.319314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:49.351025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064750b913ab4e74bd7cdb820afec6f8', 'request_id': 'req-016dbf96-65f4-498a-ab47-2a2e96c14b85'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.368692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064750b913ab4e74bd7cdb820afec6f8', 'request_id': 'req-016dbf96-65f4-498a-ab47-2a2e96c14b85'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.412845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064750b913ab4e74bd7cdb820afec6f8', 'request_id': 'req-016dbf96-65f4-498a-ab47-2a2e96c14b85'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.466353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] The token cache is being invalidate because project 064750b913ab4e74bd7cdb820afec6f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:02:49.466660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.469742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.513311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 826/1658] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:02:48 2022] DELETE /identity/v3/projects/064750b913ab4e74bd7cdb820afec6f8 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:49.516730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:49.518250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:49.549251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bc4e4-ba1a-4d94-a951-57a23ca41dc3 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:49.788716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:49.789672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:49.791585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:49.791585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] PATH_INFO: `/v3/users/8e5996e3bbcf40ca9e51b53efa90cff2` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:49.801769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5996e3bbcf40ca9e51b53efa90cff2)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:49.811622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:49.962516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5996e3bbcf40ca9e51b53efa90cff2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.963981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5996e3bbcf40ca9e51b53efa90cff2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.996649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5996e3bbcf40ca9e51b53efa90cff2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:49.997887 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5996e3bbcf40ca9e51b53efa90cff2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:50.022280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:50.025603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:50.050558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-66948441-8959-4b4e-be68-a62e6030b4aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5996e3bbcf40ca9e51b53efa90cff2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:50.057771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbf96-65f4-498a-ab47-2a2e96c14b85 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:50.072875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:50.080291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:50.085599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:50.086213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] PATH_INFO: `/v3/users/2283d984870a47f3bcf913aca5f98670` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:50.089923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2283d984870a47f3bcf913aca5f98670)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:50.091863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 833/1659] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:02:49 2022] DELETE /identity/v3/users/8e5996e3bbcf40ca9e51b53efa90cff2 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:50.093111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: WARNING py.warnings [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:02:50.093111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: warnings.warn(msg) Jun 29 08:02:50.093111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: Jun 29 08:02:50.105342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:50.262903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2283d984870a47f3bcf913aca5f98670'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:50.318682 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2283d984870a47f3bcf913aca5f98670'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:50.322675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2283d984870a47f3bcf913aca5f98670'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:50.343909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2283d984870a47f3bcf913aca5f98670'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:50.344597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2283d984870a47f3bcf913aca5f98670'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:50.391862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 827/1660] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:02:50 2022] DELETE /identity/v3/users/2283d984870a47f3bcf913aca5f98670 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:50.397401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:50.397775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:50.398210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:50.462575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:50.472162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:50.646709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 834/1661] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:50.719177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:50.724621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:50.901889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b53c5-9cc6-48a9-87f2-1097aa99e3f3 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:50.918449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e9747-80aa-4e74-9b92-03ad0deb8b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:50.919212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e9747-80aa-4e74-9b92-03ad0deb8b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:50.919508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e9747-80aa-4e74-9b92-03ad0deb8b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:50.919791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e9747-80aa-4e74-9b92-03ad0deb8b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:50.984039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e9747-80aa-4e74-9b92-03ad0deb8b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:50.985562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e9747-80aa-4e74-9b92-03ad0deb8b2a service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:51.006195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 828/1662] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:02:51.359867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:51.362503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:51.381379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162368d-34a4-463f-adbb-5ef6a67a4823 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:51.393209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:51.394533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:51.394871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:51.395168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] PATH_INFO: `/v3/projects/fa03c3fdcb7343e695f5a2be2da02465` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:51.398339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa03c3fdcb7343e695f5a2be2da02465)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:51.401311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: WARNING py.warnings [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:02:51.401311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: warnings.warn(msg) Jun 29 08:02:51.401311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: Jun 29 08:02:51.406672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:51.418323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:51.456564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0dbb9-c20b-416b-be04-f51fe0e84704 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:51.457514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0dbb9-c20b-416b-be04-f51fe0e84704 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:51.458046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0dbb9-c20b-416b-be04-f51fe0e84704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:51.526953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8cc0dbb9-c20b-416b-be04-f51fe0e84704 None None] MFA Rules not processed for user `1f4af4eedee24897be2e702ef51873b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:51.534713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8cc0dbb9-c20b-416b-be04-f51fe0e84704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:51.605726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa03c3fdcb7343e695f5a2be2da02465', 'request_id': 'req-5b89f891-93ad-4c06-964a-b022ba8bc02e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:51.620322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:51.622319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa03c3fdcb7343e695f5a2be2da02465', 'request_id': 'req-5b89f891-93ad-4c06-964a-b022ba8bc02e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:51.668341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa03c3fdcb7343e695f5a2be2da02465', 'request_id': 'req-5b89f891-93ad-4c06-964a-b022ba8bc02e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:51.697329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] The token cache is being invalidate because project fa03c3fdcb7343e695f5a2be2da02465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:02:51.697898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:51.702828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5b89f891-93ad-4c06-964a-b022ba8bc02e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:51.704954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 835/1663] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:02:51 2022] DELETE /identity/v3/projects/fa03c3fdcb7343e695f5a2be2da02465 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:51.710134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:51.710424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:51.710697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:51.720448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 829/1664] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:02:51.722279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbfd30-0dbe-4120-966a-12e4840d5f4b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:51.722541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbfd30-0dbe-4120-966a-12e4840d5f4b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:51.722829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbfd30-0dbe-4120-966a-12e4840d5f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:51.753326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:51.754584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:51.759834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-83bbfd30-0dbe-4120-966a-12e4840d5f4b None None] MFA Rules not processed for user `1f4af4eedee24897be2e702ef51873b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:51.761852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-83bbfd30-0dbe-4120-966a-12e4840d5f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:51.790567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 830/1665] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:51.848527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486b27d-b9eb-4d33-b6b0-cc685ef6cd7b None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:51.849735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486b27d-b9eb-4d33-b6b0-cc685ef6cd7b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:51.850236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486b27d-b9eb-4d33-b6b0-cc685ef6cd7b None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:51.856153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 831/1666] 10.222.0.43 () {54 vars in 961 bytes} [Wed Jun 29 08:02:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:02:51.871733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d954f1-45c4-4131-b8ae-d8b7dfe998be None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:51.873036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d954f1-45c4-4131-b8ae-d8b7dfe998be None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:51.873964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d954f1-45c4-4131-b8ae-d8b7dfe998be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:51.950235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 836/1667] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:02:51.951164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:51.959307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:51.964422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-a7d954f1-45c4-4131-b8ae-d8b7dfe998be None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:51.968515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a7d954f1-45c4-4131-b8ae-d8b7dfe998be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:51.984316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91266b4-a62f-41b9-af94-9fd9ce792552 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:52.012254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:52.018014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.018433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.018809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] PATH_INFO: `/v3/projects/50c7c44edb3c461e952d2a3f1b5da059` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.021180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c7c44edb3c461e952d2a3f1b5da059)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:52.023184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:52.152948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c7c44edb3c461e952d2a3f1b5da059', 'request_id': 'req-a30af0f0-b255-4a84-82e4-221cd0ec12fe'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.156611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 832/1668] 10.222.0.43 () {58 vars in 1058 bytes} [Wed Jun 29 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:02:52.158012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.158303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.158584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.167838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c7c44edb3c461e952d2a3f1b5da059', 'request_id': 'req-a30af0f0-b255-4a84-82e4-221cd0ec12fe'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.200493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c7c44edb3c461e952d2a3f1b5da059', 'request_id': 'req-a30af0f0-b255-4a84-82e4-221cd0ec12fe'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.221645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:52.224432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.228082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] The token cache is being invalidate because project 50c7c44edb3c461e952d2a3f1b5da059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:02:52.228391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.229117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.236409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 837/1669] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:02:51 2022] DELETE /identity/v3/projects/50c7c44edb3c461e952d2a3f1b5da059 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:52.237498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:52.239700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.273676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:52.341439 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 833/1670] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:52.344915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:52.347922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.376006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8fa17-e50b-4c31-9ad6-e27b0bfcffc7 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:52.396162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:52.397229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.397522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.397800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] PATH_INFO: `/v3/projects/50485aa2c4f9419ba622b4a278671ec7` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.401367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50485aa2c4f9419ba622b4a278671ec7)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:52.405609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:52.414435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30af0f0-b255-4a84-82e4-221cd0ec12fe admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:52.433394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:52.434080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.434366 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.434643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.437011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.479820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50485aa2c4f9419ba622b4a278671ec7', 'request_id': 'req-2760f113-1df2-4401-8766-c532e02e58e6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.494639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:52.503817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50485aa2c4f9419ba622b4a278671ec7', 'request_id': 'req-2760f113-1df2-4401-8766-c532e02e58e6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.527313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50485aa2c4f9419ba622b4a278671ec7', 'request_id': 'req-2760f113-1df2-4401-8766-c532e02e58e6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.564346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] The token cache is being invalidate because project 50485aa2c4f9419ba622b4a278671ec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:02:52.564939 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.567424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2760f113-1df2-4401-8766-c532e02e58e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.572427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 834/1671] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:02:52 2022] DELETE /identity/v3/projects/50485aa2c4f9419ba622b4a278671ec7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:52.578181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.578482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.578875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.584858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:52.587467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:52.607204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 838/1672] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:52.610440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:52.617322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:52.619958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.638925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368c17b-fba0-4b8a-8d1f-b5e2d944c7ec service nova] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:52.720212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 835/1673] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:52.723531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:52.726285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.745445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d227a-d88d-48f8-983f-f60da4a899c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:52.747959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d227a-d88d-48f8-983f-f60da4a899c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.749517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d227a-d88d-48f8-983f-f60da4a899c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.750038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d227a-d88d-48f8-983f-f60da4a899c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.753029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d227a-d88d-48f8-983f-f60da4a899c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:52.753609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d227a-d88d-48f8-983f-f60da4a899c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:52.755483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d227a-d88d-48f8-983f-f60da4a899c1 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:52.758847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139204-18db-4a16-8fd2-2d213269c2a4 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:52.775423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:52.776344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.776633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.776916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] PATH_INFO: `/v3/users/019c15c3a392460697fd7afff02f07d0` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.781544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] RBAC: Authorizing `identity:delete_user(user_id=019c15c3a392460697fd7afff02f07d0)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:52.785447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:52.791768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 839/1674] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:52.796375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.797002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.797561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.842506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:52.847171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.873741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 840/1675] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:52.876935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:52.880622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:52.904542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019c15c3a392460697fd7afff02f07d0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.910851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a38a04-705d-482b-8dd3-849ac5151adc None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:52.925833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:52.933929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:52.934446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:52.934630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019c15c3a392460697fd7afff02f07d0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.934925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:52.935413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019c15c3a392460697fd7afff02f07d0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.937937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:52.940003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:52.953109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019c15c3a392460697fd7afff02f07d0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.953791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019c15c3a392460697fd7afff02f07d0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:52.996479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 841/1676] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:52 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:52.997686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.003705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:53.009729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 836/1677] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:02:52 2022] DELETE /identity/v3/users/019c15c3a392460697fd7afff02f07d0 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:53.014125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.018675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:53.069916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:53.164478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:53.188698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae3767-92ba-40de-8891-b5c699a3901b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.192456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e74838-21be-40ad-b922-73d4141aa466 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.199660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.201219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.201494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.202610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.203525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:53.203873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.204706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.205542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.205901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.206372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.206832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.207856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.215754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.222603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 842/1678] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:53.228821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.248998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124db02-4d23-4cb5-89d0-b80d81d11dbb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.261072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7386c76-d72c-4c14-963d-2802a3b4fa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.262189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7386c76-d72c-4c14-963d-2802a3b4fa0c admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.262760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7386c76-d72c-4c14-963d-2802a3b4fa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.263617 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7386c76-d72c-4c14-963d-2802a3b4fa0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.264628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7386c76-d72c-4c14-963d-2802a3b4fa0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.267365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7386c76-d72c-4c14-963d-2802a3b4fa0c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.269622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 837/1679] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:53 2022] POST /identity/v3/users => generated 283 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:53.273902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.296628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718da410-0928-4a14-8807-b8a5152ea1c0 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.307725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07981e6f-e0d0-43ce-9f2a-fbf502b2f4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.308867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e6f-e0d0-43ce-9f2a-fbf502b2f4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.311057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e6f-e0d0-43ce-9f2a-fbf502b2f4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.311593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e6f-e0d0-43ce-9f2a-fbf502b2f4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.312606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07981e6f-e0d0-43ce-9f2a-fbf502b2f4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.316902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07981e6f-e0d0-43ce-9f2a-fbf502b2f4cb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.317863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 843/1680] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:53 2022] POST /identity/v3/projects => generated 401 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:53.319392 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b1212-99f1-4867-b58c-ee546278d115 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.319679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b1212-99f1-4867-b58c-ee546278d115 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.321263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b1212-99f1-4867-b58c-ee546278d115 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.326785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 844/1681] 10.222.0.43 () {54 vars in 912 bytes} [Wed Jun 29 08:02:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:02:53.328019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b1212-99f1-4867-b58c-ee546278d115 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.335561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 838/1682] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:02:53.338948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c49e82-378f-43ef-80d8-adc290643844 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.341790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c49e82-378f-43ef-80d8-adc290643844 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.343412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c49e82-378f-43ef-80d8-adc290643844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.396383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-77c49e82-378f-43ef-80d8-adc290643844 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:53.398929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-77c49e82-378f-43ef-80d8-adc290643844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:53.507114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b1212-99f1-4867-b58c-ee546278d115 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.514957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.515920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.516461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.516872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.517786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.520869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.558935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 845/1683] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:53 2022] POST /identity/v3/users => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:53.560545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.573177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd166fb-b143-4e30-9a3a-9179e93ead48 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.578731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.579733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.580216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.580709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.581933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.585049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.594141 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 839/1684] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:02:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:53.595051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c49e82-378f-43ef-80d8-adc290643844 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.611002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 846/1685] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:53.613909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.616287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:53.625908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c49e82-378f-43ef-80d8-adc290643844 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.642133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.643007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.644492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.644852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.645665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.650427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.684809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 840/1686] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:53.685850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.700891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9360616-d125-4e36-8a4f-2426b279947c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.706110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.707026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.707481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.708307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] PATH_INFO: `/v3/projects/4f88544f4f9844d3911d5c353392aab3/users/e493ef12863649d1a90d06a29ff0141c/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.729974 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f88544f4f9844d3911d5c353392aab3, user_id=e493ef12863649d1a90d06a29ff0141c, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.733477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.753116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d433-4ed6-43d2-b670-ef80871a2c78 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.779186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 841/1687] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:53 2022] PUT /identity/v3/projects/4f88544f4f9844d3911d5c353392aab3/users/e493ef12863649d1a90d06a29ff0141c/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:53.781915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:53.838779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:53.920851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.926516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.926834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.927119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.955022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb2161-a130-4074-9d1e-ba61e9cb7c51 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:53.962944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255ab1b-cb1a-414a-b614-bdc319f51a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:53.964102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255ab1b-cb1a-414a-b614-bdc319f51a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.964399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255ab1b-cb1a-414a-b614-bdc319f51a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.964685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255ab1b-cb1a-414a-b614-bdc319f51a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:53.966064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255ab1b-cb1a-414a-b614-bdc319f51a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:53.969095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255ab1b-cb1a-414a-b614-bdc319f51a02 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:53.991245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 842/1688] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:53.998634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8027874-33f9-4b2e-8346-294f717f0358 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:53.998999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8027874-33f9-4b2e-8346-294f717f0358 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:53.999289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8027874-33f9-4b2e-8346-294f717f0358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:54.080706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f8027874-33f9-4b2e-8346-294f717f0358 None None] MFA Rules not processed for user `e493ef12863649d1a90d06a29ff0141c`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:54.098349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f8027874-33f9-4b2e-8346-294f717f0358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:54.115646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:54.117234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:54.145439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 847/1689] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:02:54.146327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:54.158291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:54.173761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65e62-d75a-4f6b-99aa-805712f79dc5 service glance] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:54.187942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7994d-4422-4120-9924-79f38839e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:54.188866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7994d-4422-4120-9924-79f38839e2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:54.191564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7994d-4422-4120-9924-79f38839e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:54.191938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7994d-4422-4120-9924-79f38839e2d9 admin admin] PATH_INFO: `/v3/projects/a8049e9f8c5144608fc1d3160cbf335e/users/8399b1067c654a7a83189b4aba38279f/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:54.263025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7994d-4422-4120-9924-79f38839e2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8049e9f8c5144608fc1d3160cbf335e, user_id=8399b1067c654a7a83189b4aba38279f, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:54.266976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7994d-4422-4120-9924-79f38839e2d9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:54.297302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 848/1690] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:54 2022] PUT /identity/v3/projects/a8049e9f8c5144608fc1d3160cbf335e/users/8399b1067c654a7a83189b4aba38279f/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:54.310777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d289c2-6825-4edb-868f-8223b9fbbcbf None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:54.311131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d289c2-6825-4edb-868f-8223b9fbbcbf None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:54.311417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d289c2-6825-4edb-868f-8223b9fbbcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:54.359690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b9d289c2-6825-4edb-868f-8223b9fbbcbf None None] MFA Rules not processed for user `8399b1067c654a7a83189b4aba38279f`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:54.361401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b9d289c2-6825-4edb-868f-8223b9fbbcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:54.395015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 843/1691] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:54.399473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8027874-33f9-4b2e-8346-294f717f0358 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:54.420378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8027874-33f9-4b2e-8346-294f717f0358 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:54.507726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 849/1692] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:02:54.510638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d289c2-6825-4edb-868f-8223b9fbbcbf None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:54.571103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:54.571921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:54.572312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:54.573332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:54.574167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:54.576217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:54.632667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 844/1693] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:54 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:02:54.637257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:54.644801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:54.730868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d289c2-6825-4edb-868f-8223b9fbbcbf None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:54.845667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e2da-3efc-4a43-9a2a-b79c7a81db0a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:54.859342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:54.860049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:54.860332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:54.860610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:54.870206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:54.872345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:54.902974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:54.904218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:54.904520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:54.904864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:54.906006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:54.918352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:54.956276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 845/1694] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:54 2022] POST /identity/v3/users => generated 289 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:54.959933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:54.991568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e102-2c8e-4d58-8b6f-90648cd4bb05 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:55.015610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:55.016330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.016954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.017293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.027588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 850/1695] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:54 2022] POST /identity/v3/projects => generated 399 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:55.030106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:55.032274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.034312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:55.105521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 846/1696] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:55.110050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.148920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f21cd6-f8d2-4933-966d-113c1f6a7877 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:55.218691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:55.332001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e784e9f-8899-4c64-ae3b-1bfc14dcff92 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:55.345038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133aff5f-9243-4008-a1c5-ee52f702fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:55.348081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aff5f-9243-4008-a1c5-ee52f702fd8e admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.348408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aff5f-9243-4008-a1c5-ee52f702fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.348651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aff5f-9243-4008-a1c5-ee52f702fd8e admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.349421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133aff5f-9243-4008-a1c5-ee52f702fd8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:55.351436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133aff5f-9243-4008-a1c5-ee52f702fd8e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:55.382409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabecb6-dbe3-4924-9ce4-60dd5b881b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:55.383376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabecb6-dbe3-4924-9ce4-60dd5b881b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.388875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabecb6-dbe3-4924-9ce4-60dd5b881b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.389294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabecb6-dbe3-4924-9ce4-60dd5b881b18 admin admin] PATH_INFO: `/v3/projects/945be45408e0402db7e181a6a142f5f5/users/94a787d5590d4caeb1ff4fef4a8cc8a8/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.419877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 851/1697] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:55 2022] POST /identity/v3/users => generated 306 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:55.430710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d674e6-7285-4c21-bed7-e41e61efa507 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.431019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d674e6-7285-4c21-bed7-e41e61efa507 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.431335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d674e6-7285-4c21-bed7-e41e61efa507 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.432853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 852/1698] 10.222.0.43 () {54 vars in 912 bytes} [Wed Jun 29 08:02:55 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:02:55.438576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d674e6-7285-4c21-bed7-e41e61efa507 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.448056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabecb6-dbe3-4924-9ce4-60dd5b881b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=945be45408e0402db7e181a6a142f5f5, user_id=94a787d5590d4caeb1ff4fef4a8cc8a8, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:55.451554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabecb6-dbe3-4924-9ce4-60dd5b881b18 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:55.469020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d674e6-7285-4c21-bed7-e41e61efa507 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:55.494313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:55.495872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.497538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 847/1699] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:55 2022] PUT /identity/v3/projects/945be45408e0402db7e181a6a142f5f5/users/94a787d5590d4caeb1ff4fef4a8cc8a8/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:55.498287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.498659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.500210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.500506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.500790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.501820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:55.507117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:55.533382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 853/1700] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:55.537537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.551441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87be3d-7847-46cb-86eb-0774d8a03937 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:55.609518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] MFA Rules not processed for user `45a558f9c7c84a05a90595fda7fccd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:55.610830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:55.614614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:55.667068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:55.667986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.668339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.668869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.669968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:55.671883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:55.686051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 854/1701] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:55.687162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.728265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd31c5-a6f2-4a7d-bc6d-77a7fb283958 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:55.742486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:55.744056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.744384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.744743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] PATH_INFO: `/v3/projects/ef33be1035a94aa7b5f0ea88f3f00cca/users/e8961175344944ff80c333d0ee2327a0/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.749738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 848/1702] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:02:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:55.750631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.757953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:55.784168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650550f8-3860-412e-a3d6-2c55cab1008b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:55.789877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef33be1035a94aa7b5f0ea88f3f00cca, user_id=e8961175344944ff80c333d0ee2327a0, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:55.791000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:55.791894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:55.792343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:55.792625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] PATH_INFO: `/v3/projects/ed834c17f32644d6bb89b55645dd97a4` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:55.793898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed834c17f32644d6bb89b55645dd97a4)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:55.796420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:55.799561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:55.834655 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 855/1703] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:55 2022] PUT /identity/v3/projects/ef33be1035a94aa7b5f0ea88f3f00cca/users/e8961175344944ff80c333d0ee2327a0/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:55.835549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.883774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed834c17f32644d6bb89b55645dd97a4', 'request_id': 'req-c232c33d-a7f8-4d67-85ab-3e707ff5372d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:55.899927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed834c17f32644d6bb89b55645dd97a4', 'request_id': 'req-c232c33d-a7f8-4d67-85ab-3e707ff5372d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:55.922581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed834c17f32644d6bb89b55645dd97a4', 'request_id': 'req-c232c33d-a7f8-4d67-85ab-3e707ff5372d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:55.943475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] The token cache is being invalidate because project ed834c17f32644d6bb89b55645dd97a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:02:55.943834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:55.946619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:02:55.948495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 849/1704] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:02:55 2022] DELETE /identity/v3/projects/ed834c17f32644d6bb89b55645dd97a4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:55.952535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:55.955559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:56.016084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:56.021115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2283d-a303-4b2d-8897-d128f921e81b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:56.180787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c33d-a7f8-4d67-85ab-3e707ff5372d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:56.197553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:56.200411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.201589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.202172 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.203408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:56.205578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:56.233134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 850/1705] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:56.250510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:56.253386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:56.259483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c2de9-d76f-4349-b1b3-9560ce83e19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:56.260423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2de9-d76f-4349-b1b3-9560ce83e19b admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.260704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2de9-d76f-4349-b1b3-9560ce83e19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.261006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2de9-d76f-4349-b1b3-9560ce83e19b admin admin] PATH_INFO: `/v3/projects/945be45408e0402db7e181a6a142f5f5/users/94a787d5590d4caeb1ff4fef4a8cc8a8/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.275214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c2de9-d76f-4349-b1b3-9560ce83e19b admin admin] RBAC: Authorizing `identity:create_grant(project_id=945be45408e0402db7e181a6a142f5f5, user_id=94a787d5590d4caeb1ff4fef4a8cc8a8, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:56.281002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c2de9-d76f-4349-b1b3-9560ce83e19b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:56.369265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 856/1706] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:55 2022] PUT /identity/v3/projects/945be45408e0402db7e181a6a142f5f5/users/94a787d5590d4caeb1ff4fef4a8cc8a8/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:56.374461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6932706-c439-4635-b43b-7420e826e597 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.374750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6932706-c439-4635-b43b-7420e826e597 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.375031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6932706-c439-4635-b43b-7420e826e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.458940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e6932706-c439-4635-b43b-7420e826e597 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:56.463952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e6932706-c439-4635-b43b-7420e826e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:56.514448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea898b89-33a0-4e41-9913-d121230d52f9 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:56.610185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:56.633425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 857/1707] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:56.634319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6932706-c439-4635-b43b-7420e826e597 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:56.635087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b21af3-dfe3-43df-b6c0-af0d21524b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:56.635936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b21af3-dfe3-43df-b6c0-af0d21524b3f service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.636352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b21af3-dfe3-43df-b6c0-af0d21524b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.636723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b21af3-dfe3-43df-b6c0-af0d21524b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.638922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-02b21af3-dfe3-43df-b6c0-af0d21524b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:56.652467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b21af3-dfe3-43df-b6c0-af0d21524b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:56.654240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b21af3-dfe3-43df-b6c0-af0d21524b3f service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:56.665704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 851/1708] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:56.667448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a1e1d-a36a-4188-b5be-45c2ff4329c2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.667849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a1e1d-a36a-4188-b5be-45c2ff4329c2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.668251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a1e1d-a36a-4188-b5be-45c2ff4329c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.675863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6932706-c439-4635-b43b-7420e826e597 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:56.682938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62068fb-b2b4-4b61-9bd4-cdf2ed6c44d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:56.685362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62068fb-b2b4-4b61-9bd4-cdf2ed6c44d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.686421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62068fb-b2b4-4b61-9bd4-cdf2ed6c44d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.686870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62068fb-b2b4-4b61-9bd4-cdf2ed6c44d8 admin admin] PATH_INFO: `/v3/projects/ef33be1035a94aa7b5f0ea88f3f00cca/users/e8961175344944ff80c333d0ee2327a0/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.689431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62068fb-b2b4-4b61-9bd4-cdf2ed6c44d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef33be1035a94aa7b5f0ea88f3f00cca, user_id=e8961175344944ff80c333d0ee2327a0, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:56.693979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62068fb-b2b4-4b61-9bd4-cdf2ed6c44d8 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:56.713475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-940a1e1d-a36a-4188-b5be-45c2ff4329c2 None None] MFA Rules not processed for user `94a787d5590d4caeb1ff4fef4a8cc8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:56.716227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-940a1e1d-a36a-4188-b5be-45c2ff4329c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:56.730357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 858/1709] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:56 2022] PUT /identity/v3/projects/ef33be1035a94aa7b5f0ea88f3f00cca/users/e8961175344944ff80c333d0ee2327a0/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:56.732173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.732468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.732751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.754564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:56.756339 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:56.834140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:56.848677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 852/1710] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:56.848677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340835-81c7-4368-919f-69cce845fd61 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:56.848677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340835-81c7-4368-919f-69cce845fd61 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:56.848677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340835-81c7-4368-919f-69cce845fd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:56.883831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 859/1711] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:56.885386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:56.887007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:56.908156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c2340835-81c7-4368-919f-69cce845fd61 None None] MFA Rules not processed for user `e8961175344944ff80c333d0ee2327a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:56.909664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c2340835-81c7-4368-919f-69cce845fd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:57.021669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ae698-4325-4977-a473-36e0ba084e12 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.035440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.036284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.036579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.036863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.048847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 853/1712] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:57.049670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2340835-81c7-4368-919f-69cce845fd61 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.064903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.067795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.079660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2340835-81c7-4368-919f-69cce845fd61 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.092654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.093767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.094086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.094342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.095917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:02:57.099135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.101685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.109596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 860/1713] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:57.110630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.119266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:57.140657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 854/1714] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:02:57.145167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.151078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5561d-9e83-42e6-b428-f9ad0f5adb27 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.165255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9e180-59f9-4b17-866b-ceccf255a674 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.169160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.171041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.171041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.171041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.172443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.173737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.174301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.175027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.176236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.179446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.211999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.213322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.243551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 855/1715] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:57 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:57.247214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 861/1716] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:57 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:57.259175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.302985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.338227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea836ed3-b965-4c84-a045-e2fc61d7f0e4 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.420826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78cf30-0ea5-40b9-9d3c-33f7add4a7e6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.426097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.427294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.427294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.427294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.428016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.429998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.480806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.481481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.481777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.482064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.488411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:57.491840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 856/1717] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:57 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:57.492503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.512290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502f3a8-ae46-459c-9306-a617908d352d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.524989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.528126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.530072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.530072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.535659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:57.589639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:57.659522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:57.694427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.696485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.721494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 862/1718] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:57.722303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.734370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891187a0-be58-446e-929e-63bfcc458cbf service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.737736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.740436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.741353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.741611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.742215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.742667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.743591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.745725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.767637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 863/1719] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:57.772032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.772209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 857/1720] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:57.806212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c063070-ca8e-46e5-9013-371d58667014 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.818111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2a7ec-e5e8-470f-9d12-3a9c175a5c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.818957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2a7ec-e5e8-470f-9d12-3a9c175a5c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.819310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2a7ec-e5e8-470f-9d12-3a9c175a5c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.821253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2a7ec-e5e8-470f-9d12-3a9c175a5c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.822138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2a7ec-e5e8-470f-9d12-3a9c175a5c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.825730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2a7ec-e5e8-470f-9d12-3a9c175a5c4e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.846357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 864/1721] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:57.850784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:57.870328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ff05e-c16c-4b71-ad9d-f908ad833e77 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:57.885320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f355fa3-4d46-48d0-bce2-a5c241d1bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:57.887205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f355fa3-4d46-48d0-bce2-a5c241d1bf0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.887686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f355fa3-4d46-48d0-bce2-a5c241d1bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.889395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f355fa3-4d46-48d0-bce2-a5c241d1bf0c admin admin] PATH_INFO: `/v3/projects/5562fafc680148558ce88eabb4c75871/users/20d86632b2e44672a4c254afc9de4f93/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.914273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f355fa3-4d46-48d0-bce2-a5c241d1bf0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5562fafc680148558ce88eabb4c75871, user_id=20d86632b2e44672a4c254afc9de4f93, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:57.917647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f355fa3-4d46-48d0-bce2-a5c241d1bf0c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:57.940944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 858/1722] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:57 2022] PUT /identity/v3/projects/5562fafc680148558ce88eabb4c75871/users/20d86632b2e44672a4c254afc9de4f93/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:57.956040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e5196-a46d-4e35-9a5a-f17eba1f4028 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.956877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e5196-a46d-4e35-9a5a-f17eba1f4028 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.957396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e5196-a46d-4e35-9a5a-f17eba1f4028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:57.973156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469a46-6fb6-4bd4-8b6f-a48a40f924d0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:57.973553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469a46-6fb6-4bd4-8b6f-a48a40f924d0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:57.973918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469a46-6fb6-4bd4-8b6f-a48a40f924d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.063959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3a2e5196-a46d-4e35-9a5a-f17eba1f4028 None None] MFA Rules not processed for user `20d86632b2e44672a4c254afc9de4f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:58.065307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-43469a46-6fb6-4bd4-8b6f-a48a40f924d0 None None] MFA Rules not processed for user `c70c44993ab240b1a5231df4c9d550c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:58.066829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3a2e5196-a46d-4e35-9a5a-f17eba1f4028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:58.069929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-43469a46-6fb6-4bd4-8b6f-a48a40f924d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:58.281597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 865/1723] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:58.286507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e5196-a46d-4e35-9a5a-f17eba1f4028 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:58.326562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 859/1724] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:58.356728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadc83b-9b52-4bef-8a57-5fa2d75b2aa6 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:58.357341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadc83b-9b52-4bef-8a57-5fa2d75b2aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:58.358203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadc83b-9b52-4bef-8a57-5fa2d75b2aa6 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.360128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 860/1725] 10.222.0.43 () {54 vars in 961 bytes} [Wed Jun 29 08:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:02:58.375573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55759d52-aba2-4b84-8fb2-c45728c46e51 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:58.376083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55759d52-aba2-4b84-8fb2-c45728c46e51 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:58.376251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55759d52-aba2-4b84-8fb2-c45728c46e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.410560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-55759d52-aba2-4b84-8fb2-c45728c46e51 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:58.414202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-55759d52-aba2-4b84-8fb2-c45728c46e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:58.493328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e5196-a46d-4e35-9a5a-f17eba1f4028 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:58.506502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:58.508154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:58.508825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:58.508825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.510676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:58.516275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:58.546154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 861/1726] 10.222.0.43 () {58 vars in 1058 bytes} [Wed Jun 29 08:02:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:02:58.547162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55759d52-aba2-4b84-8fb2-c45728c46e51 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:58.566166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55759d52-aba2-4b84-8fb2-c45728c46e51 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:58.582128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 866/1727] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:02:58 2022] POST /identity/v3/projects => generated 367 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:58.584071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:58.584805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:58.585172 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:58.585457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.587899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:58.610460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:58.615644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:58.640186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:58.651020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 862/1728] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:02:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:02:58.760835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e8d0-5503-426d-a4d0-0158401f7cbd admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:58.775895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:58.776674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:58.778084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:58.778658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.780585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:58.784885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:58.831718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 867/1729] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:02:58 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:02:58.835602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:58.854116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312f165-ffaf-4ec7-a17c-d4d6110608bf service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:58.864121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:58.864818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:58.865166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:58.865449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.866255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:58.874808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:58.897741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 863/1730] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:02:58.901103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:58.921243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0d429-d11f-49a5-9b9e-d2cd487aea53 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:58.932286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:58.933438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:58.933711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:58.934457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] PATH_INFO: `/v3/projects/b064f96d293e40d7a81f74172fd7e9f8/users/d1351a60259a48eea6638c893ca3919a/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:58.979820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b064f96d293e40d7a81f74172fd7e9f8, user_id=d1351a60259a48eea6638c893ca3919a, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:58.983469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:59.025148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 868/1731] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:58 2022] PUT /identity/v3/projects/b064f96d293e40d7a81f74172fd7e9f8/users/d1351a60259a48eea6638c893ca3919a/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:59.025858 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:59.044580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054825e8-bc52-4fa2-8063-3392573e8856 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:59.053661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:02:59.234449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03daf6-e2a4-487c-8782-b70acbd35d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:59.235078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03daf6-e2a4-487c-8782-b70acbd35d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:59.238457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03daf6-e2a4-487c-8782-b70acbd35d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:59.238804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03daf6-e2a4-487c-8782-b70acbd35d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:59.245922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03daf6-e2a4-487c-8782-b70acbd35d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:59.252653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03daf6-e2a4-487c-8782-b70acbd35d73 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:59.281024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 864/1732] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:02:59.282065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:02:59.300104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd45731-8040-4302-a37c-d39b1156436b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:02:59.326054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83406eff-1384-40d4-aeee-6d6218273b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:02:59.327154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406eff-1384-40d4-aeee-6d6218273b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:59.327449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406eff-1384-40d4-aeee-6d6218273b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:59.327911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406eff-1384-40d4-aeee-6d6218273b01 admin admin] PATH_INFO: `/v3/projects/b064f96d293e40d7a81f74172fd7e9f8/users/d1351a60259a48eea6638c893ca3919a/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:59.347923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83406eff-1384-40d4-aeee-6d6218273b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b064f96d293e40d7a81f74172fd7e9f8, user_id=d1351a60259a48eea6638c893ca3919a, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:02:59.358444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83406eff-1384-40d4-aeee-6d6218273b01 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:02:59.428616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 869/1733] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:02:59 2022] PUT /identity/v3/projects/b064f96d293e40d7a81f74172fd7e9f8/users/d1351a60259a48eea6638c893ca3919a/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:02:59.431375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ac99-5d41-4cf3-80de-2aa000200c9f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:59.433101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ac99-5d41-4cf3-80de-2aa000200c9f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:59.433101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ac99-5d41-4cf3-80de-2aa000200c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:59.536626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d151ac99-5d41-4cf3-80de-2aa000200c9f None None] MFA Rules not processed for user `d1351a60259a48eea6638c893ca3919a`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:59.538346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d151ac99-5d41-4cf3-80de-2aa000200c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:59.716879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ed9bc-46a3-45e6-a98b-30871d255f4d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:59.717403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ed9bc-46a3-45e6-a98b-30871d255f4d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:59.717512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ed9bc-46a3-45e6-a98b-30871d255f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:59.819308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ae0ed9bc-46a3-45e6-a98b-30871d255f4d None None] MFA Rules not processed for user `0800a22846504b0d9c49e1520a69e45b`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:59.834178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ae0ed9bc-46a3-45e6-a98b-30871d255f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:02:59.901773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 865/1734] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:02:59.918189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88354-f998-41c6-bc95-5783cbab7110 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:59.918514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88354-f998-41c6-bc95-5783cbab7110 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:59.918767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88354-f998-41c6-bc95-5783cbab7110 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:59.920212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 866/1735] 10.222.0.43 () {54 vars in 912 bytes} [Wed Jun 29 08:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:02:59.931805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231afd0f-3be9-4e55-8879-c2fbb4fbaf4b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:02:59.932052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231afd0f-3be9-4e55-8879-c2fbb4fbaf4b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:02:59.932341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231afd0f-3be9-4e55-8879-c2fbb4fbaf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:02:59.981779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-231afd0f-3be9-4e55-8879-c2fbb4fbaf4b None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:02:59.983247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-231afd0f-3be9-4e55-8879-c2fbb4fbaf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:00.093283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 870/1736] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:02:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:00.093283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c2f6-564f-42ae-846e-06206748748f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.093283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c2f6-564f-42ae-846e-06206748748f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.093283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c2f6-564f-42ae-846e-06206748748f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.094498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 871/1737] 10.222.0.43 () {54 vars in 960 bytes} [Wed Jun 29 08:03:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:03:00.096681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0c2f6-564f-42ae-846e-06206748748f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:00.181184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 867/1738] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:02:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:00.187969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85381c61-0d07-4433-921c-7488cc87f29e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.189823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85381c61-0d07-4433-921c-7488cc87f29e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.190166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85381c61-0d07-4433-921c-7488cc87f29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.275915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-85381c61-0d07-4433-921c-7488cc87f29e None None] MFA Rules not processed for user `0800a22846504b0d9c49e1520a69e45b`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:00.280422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-85381c61-0d07-4433-921c-7488cc87f29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:00.303122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 868/1739] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:00.303122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.303122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.303122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.362122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:00.363435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:00.370929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0c2f6-564f-42ae-846e-06206748748f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:00.387657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:00.395729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:00.398502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.399609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.400472 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.442080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:00.447845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:00.520239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 872/1740] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:00.522150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:00.526890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:00.552869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:00.619420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 869/1741] 10.222.0.43 () {58 vars in 1057 bytes} [Wed Jun 29 08:03:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:00.621040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:00.627452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:00.675853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24cee7-12e8-49e1-9f3d-6fa216d8e5cc None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:00.705311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:00.708148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.712110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.712572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.725663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e1ef9-9345-4fbe-b925-c3747a776b57 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:00.740287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:00.741346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.742760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:00.743678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.748704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.757226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:00.848414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 870/1742] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:00.850312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:00.854932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:00.857399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:00.877810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 873/1743] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:00.890036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:00.907395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9350bb5-26a2-4687-a936-39d51ec77f94 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:00.909748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932b3fa-a018-4de7-ae93-638605398852 service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:00.914140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eefb5f-c23a-48ff-8ef6-269e795e1aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:00.915350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eefb5f-c23a-48ff-8ef6-269e795e1aeb service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.915928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eefb5f-c23a-48ff-8ef6-269e795e1aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.916427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eefb5f-c23a-48ff-8ef6-269e795e1aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.927412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c86428-bb60-4ec2-88a5-96c92cbe05b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:00.928092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86428-bb60-4ec2-88a5-96c92cbe05b9 service placement] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:00.928379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86428-bb60-4ec2-88a5-96c92cbe05b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:00.928667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86428-bb60-4ec2-88a5-96c92cbe05b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:00.931754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c5c86428-bb60-4ec2-88a5-96c92cbe05b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:00.957523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86428-bb60-4ec2-88a5-96c92cbe05b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:00.959291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86428-bb60-4ec2-88a5-96c92cbe05b9 service placement] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:00.984746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 871/1744] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:01.092030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0ba9d-76d5-4802-bd00-f558e9eaafa6 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:01.092374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0ba9d-76d5-4802-bd00-f558e9eaafa6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:01.094200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0ba9d-76d5-4802-bd00-f558e9eaafa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:01.129961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-adc0ba9d-76d5-4802-bd00-f558e9eaafa6 None None] MFA Rules not processed for user `1f4af4eedee24897be2e702ef51873b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:01.134019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eefb5f-c23a-48ff-8ef6-269e795e1aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:01.134158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-adc0ba9d-76d5-4802-bd00-f558e9eaafa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:01.142506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eefb5f-c23a-48ff-8ef6-269e795e1aeb service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:01.156521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 874/1745] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:01.384267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 872/1746] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:02.947010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ecc25-e301-4969-9d11-9dd3a46b5862 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:02.947934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ecc25-e301-4969-9d11-9dd3a46b5862 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:02.947934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ecc25-e301-4969-9d11-9dd3a46b5862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:02.982800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c72ecc25-e301-4969-9d11-9dd3a46b5862 None None] MFA Rules not processed for user `45a558f9c7c84a05a90595fda7fccd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:02.987877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c72ecc25-e301-4969-9d11-9dd3a46b5862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:03.007446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 875/1747] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:03:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:03.224641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0ba9d-76d5-4802-bd00-f558e9eaafa6 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:03.260515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0ba9d-76d5-4802-bd00-f558e9eaafa6 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:03.272104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:03.272679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:03.274972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:03.275623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:03.336109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:03.350005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:03.366149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 873/1748] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:04.010523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c653a-06aa-404f-a009-c5b383c68473 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:04.011995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c653a-06aa-404f-a009-c5b383c68473 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:04.012709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c653a-06aa-404f-a009-c5b383c68473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:04.129758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-048c653a-06aa-404f-a009-c5b383c68473 None None] MFA Rules not processed for user `c9065575ed314f6f8f3915bf596911e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:04.131137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-048c653a-06aa-404f-a009-c5b383c68473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:04.490784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 876/1749] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:04.630463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:04.686757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e39e6b-1944-4f8e-9d3c-3f2803d43fdf service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:04.724033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b41e-b101-4a0f-beac-179ba18b6979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:04.727419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b41e-b101-4a0f-beac-179ba18b6979 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:04.727686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b41e-b101-4a0f-beac-179ba18b6979 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:04.728063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b41e-b101-4a0f-beac-179ba18b6979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:04.782375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58b41e-b101-4a0f-beac-179ba18b6979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:04.788041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58b41e-b101-4a0f-beac-179ba18b6979 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:04.841405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 874/1750] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:05.564228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c653a-06aa-404f-a009-c5b383c68473 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:05.717721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:05.811579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c653a-06aa-404f-a009-c5b383c68473 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:05.832562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:05.833834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:05.834139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:05.839670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] PATH_INFO: `/v3/users/8399b1067c654a7a83189b4aba38279f` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:05.842798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8399b1067c654a7a83189b4aba38279f)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:05.844827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:06.053214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8399b1067c654a7a83189b4aba38279f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:06.059799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8399b1067c654a7a83189b4aba38279f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:06.088691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8399b1067c654a7a83189b4aba38279f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:06.089795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8399b1067c654a7a83189b4aba38279f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:06.142822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8399b1067c654a7a83189b4aba38279f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:06.179216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 877/1751] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:05 2022] DELETE /identity/v3/users/8399b1067c654a7a83189b4aba38279f => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:06.181717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caeab9-dc15-41c4-a3c2-325133a1b9ac None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:06.183121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caeab9-dc15-41c4-a3c2-325133a1b9ac None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:06.183882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caeab9-dc15-41c4-a3c2-325133a1b9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:06.296703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c8caeab9-dc15-41c4-a3c2-325133a1b9ac None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:06.307067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c8caeab9-dc15-41c4-a3c2-325133a1b9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:06.626743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 875/1752] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:06.727889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:06.730021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:06.951669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3904-8dbc-480b-aa54-a11ef00e9aaa admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:06.977802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08793f5a-a541-45ec-bd42-66fcc45a2f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:06.999559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08793f5a-a541-45ec-bd42-66fcc45a2f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:07.000181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08793f5a-a541-45ec-bd42-66fcc45a2f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:07.000931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08793f5a-a541-45ec-bd42-66fcc45a2f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:07.006717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:07.007275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:07.007699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:07.017243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:07.048316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:07.049867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:07.067133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08793f5a-a541-45ec-bd42-66fcc45a2f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:07.074980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08793f5a-a541-45ec-bd42-66fcc45a2f1c service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:07.137657 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 878/1753] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:07.175280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 876/1754] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:03:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:07.196408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:07.196923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:07.197277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:07.241307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] MFA Rules not processed for user `c70c44993ab240b1a5231df4c9d550c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:07.242667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:07.424548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:07.458983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 879/1755] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:07.477985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:07.497223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:07.771632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a6d6b-ed38-4fa1-9fb1-bd15bd94ac2b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:07.806021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25a1b2-3888-49ae-9cdc-8d001f80a0c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:07.806994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a1b2-3888-49ae-9cdc-8d001f80a0c5 service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:07.807429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a1b2-3888-49ae-9cdc-8d001f80a0c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:07.807921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a1b2-3888-49ae-9cdc-8d001f80a0c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:07.829092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25a1b2-3888-49ae-9cdc-8d001f80a0c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:07.832448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25a1b2-3888-49ae-9cdc-8d001f80a0c5 service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:07.860759 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 877/1756] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:03:08.918596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:08.925581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:09.023007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc3588-c248-4abd-bbb1-9ee5672b0d77 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:09.078436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:09.084636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:09.095918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:09.097938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] PATH_INFO: `/v3/users/20d86632b2e44672a4c254afc9de4f93` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:09.109297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d86632b2e44672a4c254afc9de4f93)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:09.115135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:09.352842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d86632b2e44672a4c254afc9de4f93'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:09.359340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d86632b2e44672a4c254afc9de4f93'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:09.386676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d86632b2e44672a4c254afc9de4f93'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:09.391691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d86632b2e44672a4c254afc9de4f93'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:09.452944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d86632b2e44672a4c254afc9de4f93'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:09.555169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 880/1757] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:08 2022] DELETE /identity/v3/users/20d86632b2e44672a4c254afc9de4f93 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:09.561868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02509aed-80f3-4fe0-9828-b0ba8ddd334b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:09.562486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02509aed-80f3-4fe0-9828-b0ba8ddd334b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:09.562826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02509aed-80f3-4fe0-9828-b0ba8ddd334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:09.570867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:09.584047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:09.665557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-02509aed-80f3-4fe0-9828-b0ba8ddd334b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:09.671620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-02509aed-80f3-4fe0-9828-b0ba8ddd334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:09.816328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32475ffe-0e19-465b-ac46-d98f0d426d98 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:09.859775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:09.861647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:09.862697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:09.864592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] PATH_INFO: `/v3/users/e493ef12863649d1a90d06a29ff0141c` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:09.873766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e493ef12863649d1a90d06a29ff0141c)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:09.881705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:09.904263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 878/1758] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:10.135547 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e493ef12863649d1a90d06a29ff0141c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:10.145107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e493ef12863649d1a90d06a29ff0141c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:10.188658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e493ef12863649d1a90d06a29ff0141c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:10.189493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e493ef12863649d1a90d06a29ff0141c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:10.225504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:10.228398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e493ef12863649d1a90d06a29ff0141c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:10.266640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 881/1759] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:09 2022] DELETE /identity/v3/users/e493ef12863649d1a90d06a29ff0141c => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:10.272431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:10.273083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:10.274835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:10.353932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:10.359940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:10.508914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:10.608362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 879/1760] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:10.616621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:10.619936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:10.624739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:10.628544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:10.877772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a3cb1-858a-4dd3-9150-dc675eb2b851 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:10.901797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:10.904552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:10.904748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:10.905544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:10.952119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:10.957053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:11.033097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 882/1761] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:11.632134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a577712-4215-48e8-805c-2e76eed9dac0 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:11.651431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:11.656333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:11.656940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:11.657450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:11.666201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:11.819787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:11.838266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:11.925054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 880/1762] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 1301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:12.543343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:12.547742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:12.600741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6564a6-2491-42ed-a25a-6901ab21cb42 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:12.620693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:12.622861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:12.623661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:12.626214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] PATH_INFO: `/v3/projects/4f88544f4f9844d3911d5c353392aab3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:12.639704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f88544f4f9844d3911d5c353392aab3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:12.653005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:12.880885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f88544f4f9844d3911d5c353392aab3', 'request_id': 'req-d592a569-b3ab-451c-b979-c06f1f100a15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:12.898391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f88544f4f9844d3911d5c353392aab3', 'request_id': 'req-d592a569-b3ab-451c-b979-c06f1f100a15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:12.944489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f88544f4f9844d3911d5c353392aab3', 'request_id': 'req-d592a569-b3ab-451c-b979-c06f1f100a15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.053569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] The token cache is being invalidate because project 4f88544f4f9844d3911d5c353392aab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:13.056082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.060150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.066489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 883/1763] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:12 2022] DELETE /identity/v3/projects/4f88544f4f9844d3911d5c353392aab3 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:13.073846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:13.107306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63219e21-b381-4c18-8a71-b0da59313de0 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:13.512055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:13.513001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:13.513308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:13.516395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] PATH_INFO: `/v3/users/94a787d5590d4caeb1ff4fef4a8cc8a8` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:13.518515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a787d5590d4caeb1ff4fef4a8cc8a8)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:13.521334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:13.800208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a787d5590d4caeb1ff4fef4a8cc8a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.851595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a787d5590d4caeb1ff4fef4a8cc8a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.853701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a787d5590d4caeb1ff4fef4a8cc8a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.874585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a787d5590d4caeb1ff4fef4a8cc8a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.875497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a787d5590d4caeb1ff4fef4a8cc8a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:13.915653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:13.916955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 881/1764] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:13 2022] DELETE /identity/v3/users/94a787d5590d4caeb1ff4fef4a8cc8a8 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:13.921762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:13.999457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592a569-b3ab-451c-b979-c06f1f100a15 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:14.296435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:14.297750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:14.298210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:14.302054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] PATH_INFO: `/v3/projects/a8049e9f8c5144608fc1d3160cbf335e` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:14.309074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8049e9f8c5144608fc1d3160cbf335e)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:14.311527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:14.397874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:14.522846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8049e9f8c5144608fc1d3160cbf335e', 'request_id': 'req-ea8817f8-ddba-4cdf-aadb-81d5f608783a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:14.547636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8049e9f8c5144608fc1d3160cbf335e', 'request_id': 'req-ea8817f8-ddba-4cdf-aadb-81d5f608783a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:14.559288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8049e9f8c5144608fc1d3160cbf335e', 'request_id': 'req-ea8817f8-ddba-4cdf-aadb-81d5f608783a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:14.590255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] The token cache is being invalidate because project a8049e9f8c5144608fc1d3160cbf335e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:14.591947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:14.595259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ea8817f8-ddba-4cdf-aadb-81d5f608783a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:14.604964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 884/1765] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:13 2022] DELETE /identity/v3/projects/a8049e9f8c5144608fc1d3160cbf335e => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:14.606945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:14.610712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:14.660324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:14.687165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a728d31-3d6a-4647-ac10-c8c35e5eb33e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:14.928383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:14.929953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:14.929953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:14.930780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] PATH_INFO: `/v3/users/e8961175344944ff80c333d0ee2327a0` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:14.938241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8961175344944ff80c333d0ee2327a0)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:14.940293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:15.297975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8961175344944ff80c333d0ee2327a0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:15.380317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8961175344944ff80c333d0ee2327a0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:15.383715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8961175344944ff80c333d0ee2327a0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:15.423708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8961175344944ff80c333d0ee2327a0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:15.424355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8961175344944ff80c333d0ee2327a0'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:15.509493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 882/1766] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:14 2022] DELETE /identity/v3/users/e8961175344944ff80c333d0ee2327a0 => generated 0 bytes in 904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:16.270651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dee82-6468-407b-911d-f5606493627c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:16.272033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dee82-6468-407b-911d-f5606493627c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:16.272673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dee82-6468-407b-911d-f5606493627c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:16.366635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b10dee82-6468-407b-911d-f5606493627c None None] MFA Rules not processed for user `45a558f9c7c84a05a90595fda7fccd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:16.369389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b10dee82-6468-407b-911d-f5606493627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:16.605353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 885/1767] 10.222.0.48 () {58 vars in 1015 bytes} [Wed Jun 29 08:03:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:16.736964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:16.739222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:16.778771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44cb54-b548-4f51-91cd-02d9993a029c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:16.797352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:16.798866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:16.799431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:16.799932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:16.823194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:16.825271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:16.856057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 883/1768] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:17.061154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dee82-6468-407b-911d-f5606493627c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:17.064826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b10dee82-6468-407b-911d-f5606493627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:17.343447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dee82-6468-407b-911d-f5606493627c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:17.373330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:17.376352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:17.377289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:17.378362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] PATH_INFO: `/v3/projects/5562fafc680148558ce88eabb4c75871` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:17.381926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5562fafc680148558ce88eabb4c75871)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:17.411852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:17.791677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5562fafc680148558ce88eabb4c75871', 'request_id': 'req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:17.822484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:17.831611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5562fafc680148558ce88eabb4c75871', 'request_id': 'req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:17.895874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5562fafc680148558ce88eabb4c75871', 'request_id': 'req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:17.964253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] The token cache is being invalidate because project 5562fafc680148558ce88eabb4c75871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:17.964881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:17.968654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:17.978292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 886/1769] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:17 2022] DELETE /identity/v3/projects/5562fafc680148558ce88eabb4c75871 => generated 0 bytes in 918 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:17.981979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:18.016911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76693-28ef-4fc0-b017-a37230b55b22 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:18.254424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:18.255495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:18.255869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:18.256487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] PATH_INFO: `/v3/users/d1351a60259a48eea6638c893ca3919a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:18.259247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1351a60259a48eea6638c893ca3919a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:18.261535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:18.277062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:18.278917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:18.303534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:18.324614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ea267-e9dc-4bfc-94c9-968a59eb0aaa admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:18.356791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:18.358715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:18.359169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:18.359650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] PATH_INFO: `/v3/projects/945be45408e0402db7e181a6a142f5f5` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:18.363403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=945be45408e0402db7e181a6a142f5f5)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:18.365935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:18.453773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1351a60259a48eea6638c893ca3919a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.488328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '945be45408e0402db7e181a6a142f5f5', 'request_id': 'req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.506969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '945be45408e0402db7e181a6a142f5f5', 'request_id': 'req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.517820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '945be45408e0402db7e181a6a142f5f5', 'request_id': 'req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.529822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1351a60259a48eea6638c893ca3919a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.533030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1351a60259a48eea6638c893ca3919a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.555918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] The token cache is being invalidate because project 945be45408e0402db7e181a6a142f5f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:18.556608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.557342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1351a60259a48eea6638c893ca3919a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.557889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-395ffc24-d1a2-4837-bb22-540f7ab0cf6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.558089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d87104fe-2786-4be1-9240-a3ea49ba9ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1351a60259a48eea6638c893ca3919a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:18.563681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 887/1770] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:18 2022] DELETE /identity/v3/projects/945be45408e0402db7e181a6a142f5f5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:18.571577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b531ad-860f-4e29-82ff-8cfba341a7f8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:18.571986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b531ad-860f-4e29-82ff-8cfba341a7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:18.572381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b531ad-860f-4e29-82ff-8cfba341a7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:18.621763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 884/1771] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:17 2022] DELETE /identity/v3/users/d1351a60259a48eea6638c893ca3919a => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:18.649234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-d3b531ad-860f-4e29-82ff-8cfba341a7f8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:18.654698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d3b531ad-860f-4e29-82ff-8cfba341a7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:18.845037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 888/1772] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:18.866297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274c834-50ab-45e6-a8bd-61e7932a714e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:18.869470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274c834-50ab-45e6-a8bd-61e7932a714e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:18.869720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274c834-50ab-45e6-a8bd-61e7932a714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:18.954047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9274c834-50ab-45e6-a8bd-61e7932a714e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:18.956339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9274c834-50ab-45e6-a8bd-61e7932a714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:18.990935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 885/1773] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:19.004072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b531ad-860f-4e29-82ff-8cfba341a7f8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:19.052474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b531ad-860f-4e29-82ff-8cfba341a7f8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:19.098246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:19.099409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:19.099906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:19.100411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:19.102076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:19.102881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:19.138181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:19.221293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 889/1774] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:19.233908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274c834-50ab-45e6-a8bd-61e7932a714e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:19.304415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274c834-50ab-45e6-a8bd-61e7932a714e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:19.326268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:19.327537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:19.328046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:19.328543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:19.347188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:19.350961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:19.462245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 886/1775] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:19 2022] POST /identity/v3/projects => generated 385 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:19.471899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:19.518019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ccac5-8182-42ca-9614-d7703c09d017 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:19.873982 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:19.936834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:19.938284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:19.940193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:19.940764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:19.943551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:19.948452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:20.060103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:20.080535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 890/1776] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:19 2022] POST /identity/v3/users => generated 293 bytes in 610 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:20.088458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb5e00-d852-4f2a-8ea8-bc02b372e4f6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:20.110726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:20.111819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:20.112599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:20.113620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:20.116636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:20.122677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:20.156278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 887/1777] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:20.156532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:20.170078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131e515-40fa-45ea-bcc4-1ec2b944ad7b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:20.213376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0f564-50b1-4e60-8318-0219509f4766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:20.218539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0f564-50b1-4e60-8318-0219509f4766 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:20.222586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0f564-50b1-4e60-8318-0219509f4766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:20.222811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0f564-50b1-4e60-8318-0219509f4766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:20.228379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0f564-50b1-4e60-8318-0219509f4766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:20.241860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0f564-50b1-4e60-8318-0219509f4766 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:20.302086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:20.305426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 891/1778] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:20.324109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859ab09-56c1-49a3-88bc-3d1b276ca2f6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:20.371878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:20.372817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:20.373147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:20.373431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] PATH_INFO: `/v3/projects/d474fa94d97b4726b436e44dbf9161f6/users/e05ff754a193445795d47d69c93410af/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:20.464628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d474fa94d97b4726b436e44dbf9161f6, user_id=e05ff754a193445795d47d69c93410af, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:20.478800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:20.557905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 888/1779] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:20 2022] PUT /identity/v3/projects/d474fa94d97b4726b436e44dbf9161f6/users/e05ff754a193445795d47d69c93410af/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:20.586454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb9c87-8c4a-4792-bb12-259cf361acf9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:20.588141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb9c87-8c4a-4792-bb12-259cf361acf9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:20.589273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb9c87-8c4a-4792-bb12-259cf361acf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:20.790951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b2eb9c87-8c4a-4792-bb12-259cf361acf9 None None] MFA Rules not processed for user `e05ff754a193445795d47d69c93410af`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:20.793378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b2eb9c87-8c4a-4792-bb12-259cf361acf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:21.063562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 892/1780] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:21.069556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:21.207666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e227435-8868-4ef9-b3cc-211e842d49c2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:21.223534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:21.225093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:21.226327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:21.227324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:21.229011 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:21.237520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:21.326364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 889/1781] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:21 2022] POST /identity/v3/projects => generated 385 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:21.343286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb9c87-8c4a-4792-bb12-259cf361acf9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:21.365669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb9c87-8c4a-4792-bb12-259cf361acf9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:21.489529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:21.490294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:21.490591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:21.491073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:21.491907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:21.501602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:21.564877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 893/1782] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:21 2022] POST /identity/v3/users => generated 299 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:21.566337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:21.593902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dc0cf-15a5-4cfe-b81e-7c6a1f0f8323 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:21.606687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:21.619385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:21.620507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:21.625690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:21.626533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:21.631662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:21.639597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:21.676745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 890/1783] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:03:21.677457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:21.698850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66763-c8dc-49b3-8a19-c55260c3129b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:21.727126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:21.728266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:21.728552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:21.729065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] PATH_INFO: `/v3/projects/b36fde9be1c449f6bd10ad28910138d0/users/e1fc56734d784748bedce684bcb3e915/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:21.779912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36fde9be1c449f6bd10ad28910138d0, user_id=e1fc56734d784748bedce684bcb3e915, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:21.783089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:21.823907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 894/1784] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:21 2022] PUT /identity/v3/projects/b36fde9be1c449f6bd10ad28910138d0/users/e1fc56734d784748bedce684bcb3e915/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:21.825775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:21.850664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b2595-a3a3-49e8-934c-1fe5d3aa2351 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:22.111145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b67b0-f6a4-4220-aba9-946d9c3053bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:22.112736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b67b0-f6a4-4220-aba9-946d9c3053bf admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:22.113379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b67b0-f6a4-4220-aba9-946d9c3053bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:22.113885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b67b0-f6a4-4220-aba9-946d9c3053bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:22.115575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b67b0-f6a4-4220-aba9-946d9c3053bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:22.119239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b67b0-f6a4-4220-aba9-946d9c3053bf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:22.141440 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 891/1785] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:22.146881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:22.182869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dffe9-2f81-42ef-a4f6-5ed97bfe2561 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:22.199305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:22.199663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:22.200053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:22.206580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c9f88-7afa-4c19-8004-91d417793ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:22.207971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c9f88-7afa-4c19-8004-91d417793ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:22.208397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c9f88-7afa-4c19-8004-91d417793ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:22.208875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c9f88-7afa-4c19-8004-91d417793ab2 admin admin] PATH_INFO: `/v3/projects/b36fde9be1c449f6bd10ad28910138d0/users/e1fc56734d784748bedce684bcb3e915/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:22.223759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c9f88-7afa-4c19-8004-91d417793ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36fde9be1c449f6bd10ad28910138d0, user_id=e1fc56734d784748bedce684bcb3e915, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:22.260597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] MFA Rules not processed for user `c9065575ed314f6f8f3915bf596911e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:22.263147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:22.278526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c9f88-7afa-4c19-8004-91d417793ab2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:22.384353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 895/1786] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:22 2022] PUT /identity/v3/projects/b36fde9be1c449f6bd10ad28910138d0/users/e1fc56734d784748bedce684bcb3e915/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:22.391492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:22.391988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:22.392445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:22.556598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] MFA Rules not processed for user `e1fc56734d784748bedce684bcb3e915`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:22.559064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:22.576435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:22.578542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 892/1787] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:22.734951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:22.737416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:22.786379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aaa0b-b6ab-4f28-aba0-9be87c0a62b7 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:22.835231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 896/1788] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:22.850064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:22.853740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:23.053779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:23.055809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.056539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.057246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] PATH_INFO: `/v3/projects/ef33be1035a94aa7b5f0ea88f3f00cca` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.061073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef33be1035a94aa7b5f0ea88f3f00cca)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:23.066230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:23.199680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57578f2b-008c-43b9-8eeb-6b464949ebd6 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:23.212723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:23.289825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef33be1035a94aa7b5f0ea88f3f00cca', 'request_id': 'req-4f725e27-3e11-464b-b9e5-33d8da69181f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:23.333774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef33be1035a94aa7b5f0ea88f3f00cca', 'request_id': 'req-4f725e27-3e11-464b-b9e5-33d8da69181f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:23.358054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef33be1035a94aa7b5f0ea88f3f00cca', 'request_id': 'req-4f725e27-3e11-464b-b9e5-33d8da69181f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:23.386853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] The token cache is being invalidate because project ef33be1035a94aa7b5f0ea88f3f00cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:23.387082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:23.390063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4f725e27-3e11-464b-b9e5-33d8da69181f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:23.394793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 893/1789] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:22 2022] DELETE /identity/v3/projects/ef33be1035a94aa7b5f0ea88f3f00cca => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:23.401408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4d7ce-d485-42fe-8f8e-a3275c410449 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.401801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4d7ce-d485-42fe-8f8e-a3275c410449 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.401976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4d7ce-d485-42fe-8f8e-a3275c410449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.430853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:23.433342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.433725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.435424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.442478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:23.478239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-dad4d7ce-d485-42fe-8f8e-a3275c410449 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:23.492618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-dad4d7ce-d485-42fe-8f8e-a3275c410449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:23.651191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:23.652807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:23.676142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 894/1790] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:23.677982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb78de-dcfc-4c8b-8018-9ae6ffbc0313 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.678280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb78de-dcfc-4c8b-8018-9ae6ffbc0313 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.678565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb78de-dcfc-4c8b-8018-9ae6ffbc0313 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.682140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 895/1791] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:03:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:03:23.683522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.685308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.685308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.689645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 897/1792] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 841 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:23.719560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:23.728522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:23.743719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 896/1793] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:23.748373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:23.773419 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b915b33-3f5a-4d43-8579-e42d9720e3da service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:23.795151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a51825-e795-43ee-bae2-13ab96eea6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:23.796281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a51825-e795-43ee-bae2-13ab96eea6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.796720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a51825-e795-43ee-bae2-13ab96eea6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.797241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a51825-e795-43ee-bae2-13ab96eea6e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.798378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a51825-e795-43ee-bae2-13ab96eea6e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:23.801847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a51825-e795-43ee-bae2-13ab96eea6e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:23.804305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a51825-e795-43ee-bae2-13ab96eea6e4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:23.813574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:23.839034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:23.840693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:23.858943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 898/1794] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:23.861450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043521-38a2-40d1-ba99-495309d77e3b None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.867078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043521-38a2-40d1-ba99-495309d77e3b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.868004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043521-38a2-40d1-ba99-495309d77e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.874474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b72a1-7915-4259-b664-59f7b0c5bb16 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:23.904697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:23.905786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:23.905932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:23.906202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] PATH_INFO: `/v3/projects/b064f96d293e40d7a81f74172fd7e9f8` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:23.922696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b064f96d293e40d7a81f74172fd7e9f8)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:23.927820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:23.954693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-fd043521-38a2-40d1-ba99-495309d77e3b None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:23.956540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fd043521-38a2-40d1-ba99-495309d77e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:24.108491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b064f96d293e40d7a81f74172fd7e9f8', 'request_id': 'req-3cff0d35-5767-4b72-8583-a3c07271cf83'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:24.128398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b064f96d293e40d7a81f74172fd7e9f8', 'request_id': 'req-3cff0d35-5767-4b72-8583-a3c07271cf83'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:24.159411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b064f96d293e40d7a81f74172fd7e9f8', 'request_id': 'req-3cff0d35-5767-4b72-8583-a3c07271cf83'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:24.191509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] The token cache is being invalidate because project b064f96d293e40d7a81f74172fd7e9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:24.191863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:24.194080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:24.209324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 897/1795] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:23 2022] DELETE /identity/v3/projects/b064f96d293e40d7a81f74172fd7e9f8 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:24.209860 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:24.217743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:24.222631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 899/1796] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:24.225561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f2431-bdda-4764-9e2e-3a77c8c79ecd None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:24.225913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f2431-bdda-4764-9e2e-3a77c8c79ecd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:24.226462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f2431-bdda-4764-9e2e-3a77c8c79ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:24.274892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f25f2431-bdda-4764-9e2e-3a77c8c79ecd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:24.278701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f25f2431-bdda-4764-9e2e-3a77c8c79ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:24.375342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:24.449693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 900/1797] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:24.459266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac3ab-892b-4429-848d-36388e89bf25 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:24.459561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac3ab-892b-4429-848d-36388e89bf25 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:24.459848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac3ab-892b-4429-848d-36388e89bf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:24.498569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-114ac3ab-892b-4429-848d-36388e89bf25 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:24.500132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-114ac3ab-892b-4429-848d-36388e89bf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:24.514012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff0d35-5767-4b72-8583-a3c07271cf83 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:24.541282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7373080-a808-42e1-adff-5c4e90649e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:24.545296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 901/1798] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:24.545874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7373080-a808-42e1-adff-5c4e90649e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:24.547681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee45e-5a44-48fa-99ec-b8004b847e0f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:24.548169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee45e-5a44-48fa-99ec-b8004b847e0f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:24.548628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee45e-5a44-48fa-99ec-b8004b847e0f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:24.550739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7373080-a808-42e1-adff-5c4e90649e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:24.551899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7373080-a808-42e1-adff-5c4e90649e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:24.552085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 902/1799] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:03:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:03:24.553106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee45e-5a44-48fa-99ec-b8004b847e0f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:24.555745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7373080-a808-42e1-adff-5c4e90649e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:24.573653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7373080-a808-42e1-adff-5c4e90649e22 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:24.591943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee45e-5a44-48fa-99ec-b8004b847e0f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:24.597499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:24.598251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:24.598570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:24.598882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:24.599706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:24.600226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:24.602456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:24.640807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 903/1800] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:24.645667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:24.660587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7f012-b7b2-4238-a097-d86dc8b27ff6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:24.680043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:24.681111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:24.681412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:24.681849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:24.682539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:24.684637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:24.686551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 898/1801] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:24 2022] POST /identity/v3/projects => generated 387 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:24.688094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064691b1-9ce1-40aa-aee2-935f9583af9f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:24.690703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064691b1-9ce1-40aa-aee2-935f9583af9f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:24.691242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064691b1-9ce1-40aa-aee2-935f9583af9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:24.770504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 904/1802] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:24 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:24.772648 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-064691b1-9ce1-40aa-aee2-935f9583af9f None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:24.775050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:24.780555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-064691b1-9ce1-40aa-aee2-935f9583af9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:24.946001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:25.018791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400fd25-9315-4ded-bcaa-01fab64822bc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.046961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.048604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.053325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.053908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.060030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.071688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.128720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 899/1803] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:03:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:25.133368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064691b1-9ce1-40aa-aee2-935f9583af9f None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.166903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 905/1804] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:24 2022] POST /identity/v3/users => generated 294 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:25.169762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.187968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064691b1-9ce1-40aa-aee2-935f9583af9f None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.204171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.205407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.205738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c2e9-8864-4584-87a8-2a2c557fa0b0 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.209169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.209169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.209169 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.213688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.218395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.219874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.220415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.220907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.223744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.230479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.265778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 906/1805] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:25.275571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.281308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 900/1806] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:25 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:25.286807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.321036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ea1ee-e6ab-46cc-b594-b6065036ee11 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.333494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.335124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.335944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.336884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.338877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.344558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.350729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b8db2-1a47-4e7f-8ec1-6a4a72cd7ce7 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.370162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.371859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.373586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.374528 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.376033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.388036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.402344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 907/1807] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:25.407736 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.410184 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:25.428105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372bc4-b44c-43d5-bfce-86fa69bb21c4 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.435970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.437581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.439877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.441406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] PATH_INFO: `/v3/users/e05ff754a193445795d47d69c93410af` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.444514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05ff754a193445795d47d69c93410af)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.447271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 901/1808] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:25.448670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.450933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.495690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92089c1-ad36-484d-944f-57ed6f5d6b93 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.527701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.528955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.529550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.530065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] PATH_INFO: `/v3/projects/428f947de2f44d3f958711de321e3ba7/users/f6018d4b0732414c94ac00d5b930daf6/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.572361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428f947de2f44d3f958711de321e3ba7, user_id=f6018d4b0732414c94ac00d5b930daf6, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.579585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.588152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05ff754a193445795d47d69c93410af'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:25.588448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05ff754a193445795d47d69c93410af'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:25.610152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05ff754a193445795d47d69c93410af'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:25.610912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05ff754a193445795d47d69c93410af'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:25.612515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 902/1809] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:25 2022] PUT /identity/v3/projects/428f947de2f44d3f958711de321e3ba7/users/f6018d4b0732414c94ac00d5b930daf6/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:25.613782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.625031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:25.657939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-039a0aca-657f-4841-9f9f-b3fd65f64702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05ff754a193445795d47d69c93410af'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:25.685906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 908/1810] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:25 2022] DELETE /identity/v3/users/e05ff754a193445795d47d69c93410af => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:25.688660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed5eeb-5cba-4760-8a24-81d4f14058eb None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.689600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed5eeb-5cba-4760-8a24-81d4f14058eb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.690126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed5eeb-5cba-4760-8a24-81d4f14058eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.761348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-d2ed5eeb-5cba-4760-8a24-81d4f14058eb None None] MFA Rules not processed for user `f6018d4b0732414c94ac00d5b930daf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:25.763490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d2ed5eeb-5cba-4760-8a24-81d4f14058eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:25.775833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe49e-8059-49a4-9c0a-0247a934a4c5 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:25.830998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:25.931104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 909/1811] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:25.937117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.937410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.937692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.955055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:25.956239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:25.956720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:25.957225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:25.958201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:25.960100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:25.961610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:25.964093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:25.978226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 903/1812] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:25.978778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 910/1813] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:25.979273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.979739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:25.988232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:25.992726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:26.012316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7e3d0-40e9-49ac-afb5-d5b7b4e8f3bb None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:26.028562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:26.036542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:26.037035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:26.037465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:26.038259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:26.042655 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:26.048811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:26.097901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 911/1814] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:25 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:26.098640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:26.203577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa384-d1d4-4b7d-aa5a-74286a2ede2c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:26.251830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a7eed-55be-4aa2-b39b-d6675e924604 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:26.263255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:26.267273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:26.267273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:26.267273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] PATH_INFO: `/v3/projects/4f33904a6b474414bd4632b2b024025c/users/2c94b4b5a48a4a2bbbac9d1e916e09dd/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:26.301930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f33904a6b474414bd4632b2b024025c, user_id=2c94b4b5a48a4a2bbbac9d1e916e09dd, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:26.305132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:26.342925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 912/1815] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:26 2022] PUT /identity/v3/projects/4f33904a6b474414bd4632b2b024025c/users/2c94b4b5a48a4a2bbbac9d1e916e09dd/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:26.343971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:26.351603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:26.385740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:26.388872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:26.390262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:26.390850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:26.403320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:26.427913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:26.431125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:26.480370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 904/1816] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:26.481818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:26.617821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4aa116-9def-4830-8f9f-742f845f0a75 service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:26.627151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1d15c-be80-4e18-979c-f69981cdc915 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:26.664094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:26.665699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:26.666208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:26.666717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:26.899833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:27.122899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994fcc4-73a8-4e8b-9b4b-319289fba23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:27.124367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994fcc4-73a8-4e8b-9b4b-319289fba23d admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:27.126477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994fcc4-73a8-4e8b-9b4b-319289fba23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:27.127151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994fcc4-73a8-4e8b-9b4b-319289fba23d admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:27.128570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994fcc4-73a8-4e8b-9b4b-319289fba23d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:27.147157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994fcc4-73a8-4e8b-9b4b-319289fba23d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:27.231099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:27.234053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:27.255577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 905/1817] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:26 2022] POST /identity/v3/users => generated 301 bytes in 772 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:27.266724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f7270-353f-49e6-9737-d3a11b46741c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:27.267419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f7270-353f-49e6-9737-d3a11b46741c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:27.268051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f7270-353f-49e6-9737-d3a11b46741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:27.308036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 913/1818] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 965 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:27.310310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:27.338078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-464f7270-353f-49e6-9737-d3a11b46741c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:27.339709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-464f7270-353f-49e6-9737-d3a11b46741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:27.351951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296f660-07f3-4cc3-ad16-a7e603c54c97 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:27.376811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:27.378456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:27.379046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 906/1819] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:27.384957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:27.386044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:27.387724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:27.394861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:27.440599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 914/1820] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:27.442027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f7270-353f-49e6-9737-d3a11b46741c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:27.459489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:27.467015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:27.470358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f7270-353f-49e6-9737-d3a11b46741c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:27.489755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:27.489755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:27.489755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:27.489755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] PATH_INFO: `/v3/projects/ca78026c6ce543299ba4ea20941f4ece/users/042460be285141d8a754a42a8a491411/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:27.515482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca78026c6ce543299ba4ea20941f4ece, user_id=042460be285141d8a754a42a8a491411, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:27.518998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:27.532301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:27.551691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 907/1821] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:27 2022] PUT /identity/v3/projects/ca78026c6ce543299ba4ea20941f4ece/users/042460be285141d8a754a42a8a491411/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:27.553550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:27.887378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29976c3-1f4b-439a-8359-c9489c159946 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:27.898389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:27.900413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:27.903084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:27.911606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:27.913437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:27.937795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:27.962516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db612d8b-244f-4d22-b0ff-f1b5e9392312 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:27.994551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 908/1822] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:27.995631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:27.999966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:28.180490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884314-8fdb-480f-a8e6-9fbe863e1d38 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:28.181290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:28.182816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:28.183568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:28.185242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:28.186139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:28.188619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:28.188619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:28.188619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:28.202046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:28.203806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:28.204363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:28.223002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 909/1823] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:28.224433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:28.239900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5db5b8-7a8c-457b-8309-32d709bdb4ca service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:28.249151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db12ff-63a8-48bf-b4b6-d2c48ba33cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:28.250322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db12ff-63a8-48bf-b4b6-d2c48ba33cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:28.250732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:28.251509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db12ff-63a8-48bf-b4b6-d2c48ba33cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:28.251509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db12ff-63a8-48bf-b4b6-d2c48ba33cf5 admin admin] PATH_INFO: `/v3/projects/ca78026c6ce543299ba4ea20941f4ece/users/042460be285141d8a754a42a8a491411/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:28.254890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db12ff-63a8-48bf-b4b6-d2c48ba33cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca78026c6ce543299ba4ea20941f4ece, user_id=042460be285141d8a754a42a8a491411, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:28.256570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:28.260614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db12ff-63a8-48bf-b4b6-d2c48ba33cf5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:28.293104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 910/1824] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:28 2022] PUT /identity/v3/projects/ca78026c6ce543299ba4ea20941f4ece/users/042460be285141d8a754a42a8a491411/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:28.304430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102030b-4426-4d6d-a19d-b055623c3b3a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:28.305314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102030b-4426-4d6d-a19d-b055623c3b3a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:28.305607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102030b-4426-4d6d-a19d-b055623c3b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:28.328898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 915/1825] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 871 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:28.409838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9102030b-4426-4d6d-a19d-b055623c3b3a None None] MFA Rules not processed for user `042460be285141d8a754a42a8a491411`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:28.421439 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9102030b-4426-4d6d-a19d-b055623c3b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:28.464070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:28.717475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 911/1826] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:28.754489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:28.758921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:28.961032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55cd89-4df6-4be7-90a7-8e806b147a1e service nova] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:29.110706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:29.142761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc241898-bd60-4c66-8cdc-ef37082035f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:29.143525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc241898-bd60-4c66-8cdc-ef37082035f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:29.145931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc241898-bd60-4c66-8cdc-ef37082035f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:29.146320 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc241898-bd60-4c66-8cdc-ef37082035f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:29.233917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc241898-bd60-4c66-8cdc-ef37082035f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:29.243882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc241898-bd60-4c66-8cdc-ef37082035f5 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:29.292709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 916/1827] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:29.705037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102030b-4426-4d6d-a19d-b055623c3b3a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:29.870846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102030b-4426-4d6d-a19d-b055623c3b3a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:29.898644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:29.900788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:29.901311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:29.901753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] PATH_INFO: `/v3/users/f6018d4b0732414c94ac00d5b930daf6` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:29.903721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6018d4b0732414c94ac00d5b930daf6)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:29.907078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:30.010270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6018d4b0732414c94ac00d5b930daf6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.070361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6018d4b0732414c94ac00d5b930daf6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.071939 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6018d4b0732414c94ac00d5b930daf6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.092932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6018d4b0732414c94ac00d5b930daf6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.093983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6018d4b0732414c94ac00d5b930daf6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.145908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 912/1828] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:29 2022] DELETE /identity/v3/users/f6018d4b0732414c94ac00d5b930daf6 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:30.146753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:30.147045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:30.147325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:30.217731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:30.223276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:30.451666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 917/1829] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:30.464116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:30.466287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:30.501109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a2860-a1cb-4637-b137-25e1bfcb0f38 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:30.518089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:30.519345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:30.519982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:30.520293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] PATH_INFO: `/v3/projects/d474fa94d97b4726b436e44dbf9161f6` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:30.522279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d474fa94d97b4726b436e44dbf9161f6)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:30.524955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:30.550002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:30.553998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:30.583969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:30.776426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd474fa94d97b4726b436e44dbf9161f6', 'request_id': 'req-24930378-63c8-4516-b75e-68385d7120ca'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.826680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd474fa94d97b4726b436e44dbf9161f6', 'request_id': 'req-24930378-63c8-4516-b75e-68385d7120ca'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.871510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd474fa94d97b4726b436e44dbf9161f6', 'request_id': 'req-24930378-63c8-4516-b75e-68385d7120ca'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.922479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] The token cache is being invalidate because project d474fa94d97b4726b436e44dbf9161f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:30.924800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.927229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:30.942400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f82dd-40bd-40a0-9840-96b998a8cc96 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:30.942825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 913/1830] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:30 2022] DELETE /identity/v3/projects/d474fa94d97b4726b436e44dbf9161f6 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:30.947218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:30.951877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:31.111503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:31.114520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:31.115007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:31.119131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:31.122749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24930378-63c8-4516-b75e-68385d7120ca admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:31.124584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:31.137596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:31.140724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:31.142578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:31.143070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] PATH_INFO: `/v3/users/e1fc56734d784748bedce684bcb3e915` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:31.147447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fc56734d784748bedce684bcb3e915)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:31.157651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:31.178537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:31.180923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:31.204009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 918/1831] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:31.363022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fc56734d784748bedce684bcb3e915'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:31.410500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fc56734d784748bedce684bcb3e915'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:31.411809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fc56734d784748bedce684bcb3e915'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:31.437406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fc56734d784748bedce684bcb3e915'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:31.438611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0c046f99-7e39-4a45-8563-3d18628df4ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fc56734d784748bedce684bcb3e915'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:31.489704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 914/1832] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:30 2022] DELETE /identity/v3/users/e1fc56734d784748bedce684bcb3e915 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:32.358198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:32.383839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8ca65-96f6-4e2a-aafa-0f11bfbdd020 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:32.480437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:32.506677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:32.508722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:32.509051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:32.509560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] PATH_INFO: `/v3/projects/b36fde9be1c449f6bd10ad28910138d0` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:32.511290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36fde9be1c449f6bd10ad28910138d0)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:32.513744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:32.582360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:32.582892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:32.583388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:32.592827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36fde9be1c449f6bd10ad28910138d0', 'request_id': 'req-821171f5-916c-4c06-9ffe-863b1d66e2a7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:32.605664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36fde9be1c449f6bd10ad28910138d0', 'request_id': 'req-821171f5-916c-4c06-9ffe-863b1d66e2a7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:32.619101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36fde9be1c449f6bd10ad28910138d0', 'request_id': 'req-821171f5-916c-4c06-9ffe-863b1d66e2a7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:32.644919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] MFA Rules not processed for user `2c94b4b5a48a4a2bbbac9d1e916e09dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:32.647317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:32.649044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] The token cache is being invalidate because project b36fde9be1c449f6bd10ad28910138d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:32.649471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:32.651145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-821171f5-916c-4c06-9ffe-863b1d66e2a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:32.657884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 919/1833] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:32 2022] DELETE /identity/v3/projects/b36fde9be1c449f6bd10ad28910138d0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:32.662727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:32.664723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:32.665529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:32.692083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:32.713253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:32.716498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:32.805906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 915/1834] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:32.805906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:32.808556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:32.857240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 920/1835] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:32.858829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:32.863870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:32.898357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f422c-e15b-4794-95c8-73e0dbffd1f6 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:32.911440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fb2de-3126-4c52-9baa-75fcd756f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:32.912119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fb2de-3126-4c52-9baa-75fcd756f20f admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:32.912400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fb2de-3126-4c52-9baa-75fcd756f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:32.912678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fb2de-3126-4c52-9baa-75fcd756f20f admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:32.913972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fb2de-3126-4c52-9baa-75fcd756f20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:32.916729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fb2de-3126-4c52-9baa-75fcd756f20f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:32.958615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 921/1836] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:32 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:32.958615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5a85-f717-418d-9798-1cccb555450c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:32.958615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5a85-f717-418d-9798-1cccb555450c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:32.958615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5a85-f717-418d-9798-1cccb555450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:32.973655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796687e8-718a-49ee-aa8c-9d8fcbc17270 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:32.989595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-eaaf5a85-f717-418d-9798-1cccb555450c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:32.991038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-eaaf5a85-f717-418d-9798-1cccb555450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:33.059785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.060692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.061188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.061306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] PATH_INFO: `/v3/projects/428f947de2f44d3f958711de321e3ba7` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.063170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=428f947de2f44d3f958711de321e3ba7)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.065337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.092113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:33.117170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 922/1837] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:33.118379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5a85-f717-418d-9798-1cccb555450c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.131957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5a85-f717-418d-9798-1cccb555450c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:33.137843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.139612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.139612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.139612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.139892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.141821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.176562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 923/1838] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:33 2022] POST /identity/v3/users => generated 292 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:33.177787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.192020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bbc49-a5ab-4a04-9f04-8478ebbf701c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:33.193499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428f947de2f44d3f958711de321e3ba7', 'request_id': 'req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.218990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428f947de2f44d3f958711de321e3ba7', 'request_id': 'req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.233785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428f947de2f44d3f958711de321e3ba7', 'request_id': 'req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.239378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:33.265587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] The token cache is being invalidate because project 428f947de2f44d3f958711de321e3ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:33.267091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.269806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.278740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 916/1839] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:32 2022] DELETE /identity/v3/projects/428f947de2f44d3f958711de321e3ba7 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:33.278740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.278740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:33.291896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.292653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.292948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.293261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.294027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:33.294396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.296272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.315024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 924/1840] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:33.316078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.317921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:33.360683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5515d7-6bae-47f1-bcf1-6426b550db4f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:33.377076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.378778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.379308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.379723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] PATH_INFO: `/v3/users/042460be285141d8a754a42a8a491411` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.382849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042460be285141d8a754a42a8a491411)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.393079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.406236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcb394-a49c-4eb2-a4b3-8c815bd2b6ed admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:33.412637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.414224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.414786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.415275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.416231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.418780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.440386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 917/1841] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:33.443007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.446223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:33.477026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8050c-3f44-458e-aa10-9815821c6cad admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:33.513204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042460be285141d8a754a42a8a491411'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.514387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042460be285141d8a754a42a8a491411'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.522121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.523789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.524289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.524830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.528274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.541010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.548775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042460be285141d8a754a42a8a491411'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.549899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042460be285141d8a754a42a8a491411'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.590191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042460be285141d8a754a42a8a491411'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:33.623548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 925/1842] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:33 2022] DELETE /identity/v3/users/042460be285141d8a754a42a8a491411 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:33.625914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.628693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:33.637204 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 918/1843] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:33 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:33.638252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.647658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:33.762358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:33.793343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844f0ee-ff6f-430c-99f5-dabbfc8027b5 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:33.803133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dade8-2318-4407-97ec-65c459819862 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:33.804399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.805918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.806819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.807008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.808490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.812070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:33.812817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.813720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:33.814199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:33.814665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] PATH_INFO: `/v3/projects/bb208e395a2e4fb7b16a7e21dba1e573/users/d7632ef499014cde8c7f2610040b521e/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:33.842494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb208e395a2e4fb7b16a7e21dba1e573, user_id=d7632ef499014cde8c7f2610040b521e, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:33.846803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:33.855302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 919/1844] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:33 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:33.863545 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.870689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 926/1845] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:33 2022] PUT /identity/v3/projects/bb208e395a2e4fb7b16a7e21dba1e573/users/d7632ef499014cde8c7f2610040b521e/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:33.877945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:33.883623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762178fc-9a7d-439f-bab4-dc2b056adf2d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.006493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847e1be-a0fb-4cf2-964a-6cf81fa82ce7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.011215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.012284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.012768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.013441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.014516 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.014844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.015353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.015704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.016045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.016879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.018010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.018725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.020751 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:34.044775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 927/1846] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:03:34.050288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 920/1847] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:03:34.050696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:34.062187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:34.079833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8c915-889a-41c4-8a11-551fa90e88d6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.087740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40c761-a4ed-44fc-9ff0-6a09bdf4f371 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.093826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.094667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.095109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.095448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.096039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.096585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.098672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.098879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.099650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.100390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] PATH_INFO: `/v3/projects/bb208e395a2e4fb7b16a7e21dba1e573/users/d7632ef499014cde8c7f2610040b521e/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.107070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb208e395a2e4fb7b16a7e21dba1e573, user_id=d7632ef499014cde8c7f2610040b521e, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.113784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 921/1848] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:34.114454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.119097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:34.133436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9004-d4fe-4cb8-934e-e6b9a02786ff admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.139842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdf741-6af4-465b-a635-141efb984bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.140689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdf741-6af4-465b-a635-141efb984bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.140972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdf741-6af4-465b-a635-141efb984bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.141292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdf741-6af4-465b-a635-141efb984bdb admin admin] PATH_INFO: `/v3/projects/e9cdb70897dd483c817348e1e73fd314/users/0f7b7594a00f47778d912bea94ba831b/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.151899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 928/1849] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:34 2022] PUT /identity/v3/projects/bb208e395a2e4fb7b16a7e21dba1e573/users/d7632ef499014cde8c7f2610040b521e/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:34.155966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:34.160090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdf741-6af4-465b-a635-141efb984bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cdb70897dd483c817348e1e73fd314, user_id=0f7b7594a00f47778d912bea94ba831b, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.163705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdf741-6af4-465b-a635-141efb984bdb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.168284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:34.196162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 922/1850] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:34 2022] PUT /identity/v3/projects/e9cdb70897dd483c817348e1e73fd314/users/0f7b7594a00f47778d912bea94ba831b/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:34.199296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69fcca-0bcb-41cd-955c-93d592ad4d99 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.199801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69fcca-0bcb-41cd-955c-93d592ad4d99 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.200340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69fcca-0bcb-41cd-955c-93d592ad4d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.255118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7f69fcca-0bcb-41cd-955c-93d592ad4d99 None None] MFA Rules not processed for user `0f7b7594a00f47778d912bea94ba831b`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:34.257897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7f69fcca-0bcb-41cd-955c-93d592ad4d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:34.321427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcb804-18b2-4112-83ff-815d4c9907d0 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.423526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.424642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.424864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.425301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] PATH_INFO: `/v3/projects/ca78026c6ce543299ba4ea20941f4ece` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.427306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca78026c6ce543299ba4ea20941f4ece)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.429361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.433127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:34.451261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 923/1851] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:34.456815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69fcca-0bcb-41cd-955c-93d592ad4d99 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:34.483959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69fcca-0bcb-41cd-955c-93d592ad4d99 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.495864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.497712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.498740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.499217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.500940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.508064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.519787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca78026c6ce543299ba4ea20941f4ece', 'request_id': 'req-556b890b-3310-45a3-a0cd-314352b87f34'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:34.528845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca78026c6ce543299ba4ea20941f4ece', 'request_id': 'req-556b890b-3310-45a3-a0cd-314352b87f34'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:34.539199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca78026c6ce543299ba4ea20941f4ece', 'request_id': 'req-556b890b-3310-45a3-a0cd-314352b87f34'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:34.583905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 924/1852] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:34 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:34.588498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:34.589967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] The token cache is being invalidate because project ca78026c6ce543299ba4ea20941f4ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:34.590742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:34.592645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-556b890b-3310-45a3-a0cd-314352b87f34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:34.599075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 929/1853] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:34 2022] DELETE /identity/v3/projects/ca78026c6ce543299ba4ea20941f4ece => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:34.611903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418f93e-4886-404c-ad6c-efa708608128 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.612277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418f93e-4886-404c-ad6c-efa708608128 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.617272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418f93e-4886-404c-ad6c-efa708608128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.698612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2418f93e-4886-404c-ad6c-efa708608128 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:34.703201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2418f93e-4886-404c-ad6c-efa708608128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:34.847489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd077c-c607-48af-8468-4126bbb5f74b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:34.857933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:34.859013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.859488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.859982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.861541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:34.865386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:34.919294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 925/1854] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:34 2022] POST /identity/v3/users => generated 299 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:34.922188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 930/1855] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:34.931876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:34.936883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:34.937385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4352f-269c-4d4c-b6c7-297a9dbd522a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:34.937694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4352f-269c-4d4c-b6c7-297a9dbd522a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:34.938556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4352f-269c-4d4c-b6c7-297a9dbd522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:34.998699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5a2e9-708d-434c-9075-b669bc013804 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:35.000679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7bf4352f-269c-4d4c-b6c7-297a9dbd522a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:35.003693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7bf4352f-269c-4d4c-b6c7-297a9dbd522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:35.021559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:35.027291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.027827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.029815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.031271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:35.042786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:35.060303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 931/1856] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:35.068104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4352f-269c-4d4c-b6c7-297a9dbd522a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:35.109823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 926/1857] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:35.118472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:35.126428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4352f-269c-4d4c-b6c7-297a9dbd522a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:35.154626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:35.164470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.164470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.164470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.164470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:35.164470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:35.168887 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:35.177553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51059b70-2e21-4b4b-bfa2-61000622552f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:35.190782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:35.191777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.192065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.193293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] PATH_INFO: `/v3/projects/7396c78b63f04a2d96eb909f481c5fd9/users/11a7966763f54703953a9f902b8de1fb/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.202722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:35.203216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 932/1858] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:35.210780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:35.234943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7396c78b63f04a2d96eb909f481c5fd9, user_id=11a7966763f54703953a9f902b8de1fb, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:35.246102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:35.252315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d80ec-e935-41d5-b4c4-44fb4e0a6bc3 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:35.268264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:35.272251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.272614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.278753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.278753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:35.281005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:35.290539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 927/1859] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:35 2022] PUT /identity/v3/projects/7396c78b63f04a2d96eb909f481c5fd9/users/11a7966763f54703953a9f902b8de1fb/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:35.299928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:35.452056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 933/1860] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:35 2022] POST /identity/v3/projects => generated 377 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:35.459720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:35.602690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c0859-8b5e-4682-bb21-e61609cfbc69 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:35.653726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:35.706224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fbd3e-a557-4a40-921a-867e218429ba admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:35.716378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:35.717416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.718244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.718732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.720029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:35.720346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:35.720834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.721417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.721796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.722724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:35.724606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:35.728632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:35.756452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 928/1861] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:35.758941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:35.781760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bde41-7121-4114-98f7-8479af88f65d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:35.793305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bf6d0-7ad2-47a0-9e4f-6607a43331f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:35.797692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf6d0-7ad2-47a0-9e4f-6607a43331f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.801239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf6d0-7ad2-47a0-9e4f-6607a43331f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.802152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf6d0-7ad2-47a0-9e4f-6607a43331f1 admin admin] PATH_INFO: `/v3/projects/7396c78b63f04a2d96eb909f481c5fd9/users/11a7966763f54703953a9f902b8de1fb/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.807909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bf6d0-7ad2-47a0-9e4f-6607a43331f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7396c78b63f04a2d96eb909f481c5fd9, user_id=11a7966763f54703953a9f902b8de1fb, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:35.811359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bf6d0-7ad2-47a0-9e4f-6607a43331f1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:35.837944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 929/1862] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:35 2022] PUT /identity/v3/projects/7396c78b63f04a2d96eb909f481c5fd9/users/11a7966763f54703953a9f902b8de1fb/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:35.843576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 934/1863] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:35 2022] POST /identity/v3/users => generated 289 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:35.844283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:35.844686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:35.845131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:35.847784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:35.888639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] MFA Rules not processed for user `11a7966763f54703953a9f902b8de1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:35.891467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:35.902460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bcce7-90dd-4366-89e0-1f0216822d60 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:36.104887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:36.153997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 930/1864] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:36.170501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:36.172589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:36.188136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:36.189749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:36.190363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:36.190931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:36.193150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:36.197760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:36.222908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 935/1865] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:36.230824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:36.260900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ab30b-dd17-404f-a0d8-d7d4e418497e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:36.283483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:36.283483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:36.284286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:36.285073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:36.287083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:36.291329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:36.315758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 936/1866] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:36.320632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:36.333569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37524d-2e68-4680-95e6-0e835d77b50c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:36.342981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8ffd3-d4b9-45b1-bbaf-bda37ec0159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:36.344260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8ffd3-d4b9-45b1-bbaf-bda37ec0159c admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:36.344260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8ffd3-d4b9-45b1-bbaf-bda37ec0159c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:36.344591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8ffd3-d4b9-45b1-bbaf-bda37ec0159c admin admin] PATH_INFO: `/v3/projects/b2d308e66c6947dda9e49b2196a5011a/users/5808f1c606034f09b0c240dd0c573786/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:36.348760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc282d-6cd2-446d-be33-50f57621506b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:36.357858 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:36.360871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:36.361720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:36.362303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:36.369263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:36.386696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8ffd3-d4b9-45b1-bbaf-bda37ec0159c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d308e66c6947dda9e49b2196a5011a, user_id=5808f1c606034f09b0c240dd0c573786, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:36.394316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8ffd3-d4b9-45b1-bbaf-bda37ec0159c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:36.446433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 937/1867] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:36 2022] PUT /identity/v3/projects/b2d308e66c6947dda9e49b2196a5011a/users/5808f1c606034f09b0c240dd0c573786/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:36.452716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd294b-26be-45f2-8612-cd90d5b38174 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:36.453754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd294b-26be-45f2-8612-cd90d5b38174 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:36.453754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd294b-26be-45f2-8612-cd90d5b38174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:36.526166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-dedd294b-26be-45f2-8612-cd90d5b38174 None None] MFA Rules not processed for user `5808f1c606034f09b0c240dd0c573786`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:36.527861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dedd294b-26be-45f2-8612-cd90d5b38174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:36.575635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:36.579469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:36.611739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 931/1868] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:36.646632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 938/1869] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:36.652366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:36.678644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e8154-58ed-4c19-a1be-71511fa15e08 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:36.810418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:36.811166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:36.811476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:36.811749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:36.812480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:36.815531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:36.897098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 932/1870] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:36 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:36.902649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd294b-26be-45f2-8612-cd90d5b38174 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:36.928026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd294b-26be-45f2-8612-cd90d5b38174 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:37.014734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:37.124938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4a55a-42fa-4b6d-bbf3-a7e77ee80352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:37.126272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4a55a-42fa-4b6d-bbf3-a7e77ee80352 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:37.126855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4a55a-42fa-4b6d-bbf3-a7e77ee80352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:37.127642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4a55a-42fa-4b6d-bbf3-a7e77ee80352 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:37.129577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4a55a-42fa-4b6d-bbf3-a7e77ee80352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:37.133804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4a55a-42fa-4b6d-bbf3-a7e77ee80352 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:37.179838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 939/1871] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:36 2022] POST /identity/v3/users => generated 290 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:37.186375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:37.256257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063494f-7ce4-44a0-bd79-f70aee38b6c4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:37.284118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a517e-06c0-4eeb-b93f-faa83c57606e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:37.284393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:37.284598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a517e-06c0-4eeb-b93f-faa83c57606e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:37.285052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a517e-06c0-4eeb-b93f-faa83c57606e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:37.288777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:37.288777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:37.288777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:37.290920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:37.298880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:37.363073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 933/1872] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:37.369413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:37.400934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0a1a517e-06c0-4eeb-b93f-faa83c57606e None None] MFA Rules not processed for user `d7632ef499014cde8c7f2610040b521e`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:37.405401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0a1a517e-06c0-4eeb-b93f-faa83c57606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:37.424652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd361859-3198-4871-b45a-049cd878ee76 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:37.450774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:37.452562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:37.453694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:37.454213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:37.455781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:37.459050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:37.482327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 934/1873] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:37.487200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:37.513056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61161db4-59ab-48e2-b592-c797f114b3eb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:37.525666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083923f-30bb-4799-9264-f3475dbd23fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:37.527428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083923f-30bb-4799-9264-f3475dbd23fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:37.527933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083923f-30bb-4799-9264-f3475dbd23fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:37.528423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083923f-30bb-4799-9264-f3475dbd23fd admin admin] PATH_INFO: `/v3/projects/7e07f31c725e46b0a49c664630df43cf/users/0bb3c784213a41c0a374b5746bb82409/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:37.564021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:37.579176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083923f-30bb-4799-9264-f3475dbd23fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e07f31c725e46b0a49c664630df43cf, user_id=0bb3c784213a41c0a374b5746bb82409, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:37.589578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083923f-30bb-4799-9264-f3475dbd23fd admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:37.644654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 935/1874] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:37 2022] PUT /identity/v3/projects/7e07f31c725e46b0a49c664630df43cf/users/0bb3c784213a41c0a374b5746bb82409/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:37.656013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 940/1875] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:37.665572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d72b8-ad09-400b-bb67-2817e051fd17 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:37.666073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d72b8-ad09-400b-bb67-2817e051fd17 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:37.666570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d72b8-ad09-400b-bb67-2817e051fd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:37.682187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a517e-06c0-4eeb-b93f-faa83c57606e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:37.779474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e38d72b8-ad09-400b-bb67-2817e051fd17 None None] MFA Rules not processed for user `0bb3c784213a41c0a374b5746bb82409`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:37.792730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e38d72b8-ad09-400b-bb67-2817e051fd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:37.886255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a517e-06c0-4eeb-b93f-faa83c57606e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:37.983132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 936/1876] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:37.987807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d72b8-ad09-400b-bb67-2817e051fd17 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:38.020824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:38.022493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.023571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.024249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:38.027968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:38.054517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:38.161834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:38.164904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:38.169248 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d72b8-ad09-400b-bb67-2817e051fd17 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:38.177836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:38.178494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.179018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.179468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:38.180220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:38.197126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 941/1877] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:38.198140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:38.234372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 937/1878] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:37 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:38.239082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:38.251479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a210294-e63c-4980-b32d-0ac4ac2f9b31 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:38.339731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:38.340502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.340881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.341285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:38.342619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:38.346054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:38.401939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 942/1879] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:38 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:38.403264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:38.418241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597cc87-184c-4a40-84fc-405274f1f164 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:38.424352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:38.426009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.426512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.427039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:38.427838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:38.430157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:38.448021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 938/1880] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:38.449406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:38.474596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031acc73-57db-46b9-af1d-5ee70d82186c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:38.484682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:38.487362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.487729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.488075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] PATH_INFO: `/v3/projects/85f3a1c4fb1c4006b99fc3c5bdbb81b8/users/c158eeff2f834b14b29b45c44ba4989c/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:38.529912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f3a1c4fb1c4006b99fc3c5bdbb81b8, user_id=c158eeff2f834b14b29b45c44ba4989c, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:38.542970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:38.593865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 943/1881] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:38 2022] PUT /identity/v3/projects/85f3a1c4fb1c4006b99fc3c5bdbb81b8/users/c158eeff2f834b14b29b45c44ba4989c/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:38.597871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:38.609338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37c8c8-cbf7-4df5-bcf0-2de31079f1f4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:38.689695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:38.703349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05807d-5bcf-45ee-a4a0-a23a7325be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:38.705890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05807d-5bcf-45ee-a4a0-a23a7325be21 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.706929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05807d-5bcf-45ee-a4a0-a23a7325be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.707825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05807d-5bcf-45ee-a4a0-a23a7325be21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:38.709433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05807d-5bcf-45ee-a4a0-a23a7325be21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:38.715722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05807d-5bcf-45ee-a4a0-a23a7325be21 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:38.763900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 939/1882] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:03:38.770347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:38.791952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc790ee-6b91-45b7-b97e-796f4008a254 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:38.812301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:38.816055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.816500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.823051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] PATH_INFO: `/v3/projects/85f3a1c4fb1c4006b99fc3c5bdbb81b8/users/c158eeff2f834b14b29b45c44ba4989c/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:38.837830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f3a1c4fb1c4006b99fc3c5bdbb81b8, user_id=c158eeff2f834b14b29b45c44ba4989c, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:38.841371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:38.872111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 944/1883] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:38 2022] PUT /identity/v3/projects/85f3a1c4fb1c4006b99fc3c5bdbb81b8/users/c158eeff2f834b14b29b45c44ba4989c/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:38.901004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:38.901468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:38.901998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:39.024925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] MFA Rules not processed for user `c158eeff2f834b14b29b45c44ba4989c`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:39.029164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:39.283009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 940/1884] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:39.414111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:39.627428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf347df-6ce6-4783-96e7-f2b80d5f95cc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:39.644132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:39.646575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:39.647125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:39.649284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:39.667685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:39.670263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:39.698374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 945/1885] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:39.780259 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:39.781752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:39.807295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec810956-1e16-48bd-b7f9-a9049e772564 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:39.818336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01ca68-5cd8-47c2-97da-89f0649fb257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:39.819402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01ca68-5cd8-47c2-97da-89f0649fb257 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:39.820896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01ca68-5cd8-47c2-97da-89f0649fb257 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:39.821597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01ca68-5cd8-47c2-97da-89f0649fb257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:39.986821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01ca68-5cd8-47c2-97da-89f0649fb257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:39.990042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01ca68-5cd8-47c2-97da-89f0649fb257 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:40.045711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 941/1886] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:40.067977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:40.100157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b0aa5-1086-4ca5-9322-ef3190d9c3b4 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:40.109230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e08fd9-9cda-4eb5-b35c-1af693afa5b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:40.110992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e08fd9-9cda-4eb5-b35c-1af693afa5b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:40.111945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e08fd9-9cda-4eb5-b35c-1af693afa5b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:40.112284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e08fd9-9cda-4eb5-b35c-1af693afa5b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:40.125850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:40.321050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e08fd9-9cda-4eb5-b35c-1af693afa5b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:40.324501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e08fd9-9cda-4eb5-b35c-1af693afa5b7 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:40.360649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 946/1887] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:42.155535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba57af1-ec9a-48a7-a83b-7980e2e6b76d None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:42.156380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba57af1-ec9a-48a7-a83b-7980e2e6b76d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:42.156380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba57af1-ec9a-48a7-a83b-7980e2e6b76d None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:42.159399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 942/1888] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:03:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:03:42.181370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:42.182287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:42.183158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:42.207517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:42.221688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:42.402959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 947/1889] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:03:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:42.420424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba57af1-ec9a-48a7-a83b-7980e2e6b76d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:42.422164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bba57af1-ec9a-48a7-a83b-7980e2e6b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:42.443711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba57af1-ec9a-48a7-a83b-7980e2e6b76d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:42.462904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:42.464380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:42.465252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:42.465840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:42.505156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:42.509033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:42.533836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 943/1890] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:42.693873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:42.696719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:42.837961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21990f-1433-45de-976d-84a34ff1d276 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:42.844633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:42.845779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:42.846097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:42.846456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] PATH_INFO: `/v3/users/0b66efcbfeb74648a3a52fb6df10195a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:42.847998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b66efcbfeb74648a3a52fb6df10195a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:42.851502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:42.962822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:42.983978 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b66efcbfeb74648a3a52fb6df10195a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:42.985288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b66efcbfeb74648a3a52fb6df10195a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:43.002226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b66efcbfeb74648a3a52fb6df10195a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:43.003841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b66efcbfeb74648a3a52fb6df10195a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:43.063472 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fd301b83-55eb-4745-b229-e6a7fedd3610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b66efcbfeb74648a3a52fb6df10195a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:43.078757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:43.103777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60cd3e-0312-4d2e-b072-5770d87264c5 service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:43.111001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 948/1891] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:42 2022] DELETE /identity/v3/users/0b66efcbfeb74648a3a52fb6df10195a => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:43.117038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:43.118148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:43.118900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:43.119376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:43.119511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:43.119863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:43.120262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:43.125164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:43.158469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:43.160546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:43.243962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:43.254987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:43.258632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:43.291489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 944/1892] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:43.348926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 949/1893] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:43.362640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:43.366025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:43.576689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2c34-bded-423d-bac0-f9f93fcaff77 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:43.691157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496448e-1c9a-4a61-84c7-1e6a9ebcecfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:43.692101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496448e-1c9a-4a61-84c7-1e6a9ebcecfa service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:43.692387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496448e-1c9a-4a61-84c7-1e6a9ebcecfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:43.692966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496448e-1c9a-4a61-84c7-1e6a9ebcecfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:43.708466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496448e-1c9a-4a61-84c7-1e6a9ebcecfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:43.710728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496448e-1c9a-4a61-84c7-1e6a9ebcecfa service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:43.724799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 945/1894] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:44.817042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:44.819968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:44.836084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29e26f-890f-4e81-a44b-8f9374a2613c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:44.860819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:44.863729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:44.864277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:44.866988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] PATH_INFO: `/v3/users/5808f1c606034f09b0c240dd0c573786` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:44.869617 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5808f1c606034f09b0c240dd0c573786)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:44.900966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:44.990508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5808f1c606034f09b0c240dd0c573786'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:44.992252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5808f1c606034f09b0c240dd0c573786'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.014497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5808f1c606034f09b0c240dd0c573786'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.015534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5808f1c606034f09b0c240dd0c573786'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.039840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5808f1c606034f09b0c240dd0c573786'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.067500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 950/1895] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:44 2022] DELETE /identity/v3/users/5808f1c606034f09b0c240dd0c573786 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:45.073810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:45.074456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:45.074888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:45.149690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:45.155836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:45.203975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:45.209489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:45.388171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:45.391997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 946/1896] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:45.421958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:45.435297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:45.559212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20f08a-6773-4187-82fe-c01ff6c020fa admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:45.565921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:45.567137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:45.567307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:45.567488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] PATH_INFO: `/v3/projects/61cd7366997e4508a1a553eb109695af` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:45.569559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61cd7366997e4508a1a553eb109695af)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:45.572515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:45.638583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbe8b4-a848-4e81-a54d-4afc87bd0fa7 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:45.655114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:45.659221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:45.661583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:45.663366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:45.704010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:45.709330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:45.727630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:45.744945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61cd7366997e4508a1a553eb109695af', 'request_id': 'req-a37ec87c-d173-4ec3-aecb-77358f4feed2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.769800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61cd7366997e4508a1a553eb109695af', 'request_id': 'req-a37ec87c-d173-4ec3-aecb-77358f4feed2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.791083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61cd7366997e4508a1a553eb109695af', 'request_id': 'req-a37ec87c-d173-4ec3-aecb-77358f4feed2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.794524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 947/1897] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:45.825053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] The token cache is being invalidate because project 61cd7366997e4508a1a553eb109695af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:45.825955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.828567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:45.831445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 951/1898] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:45 2022] DELETE /identity/v3/projects/61cd7366997e4508a1a553eb109695af => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:45.837637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:45.855696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b2777-b63f-48c4-908f-e7bab585bb81 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:46.131449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:46.138017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:46.139609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:46.141481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] PATH_INFO: `/v3/users/8573e7c4d2d4404b9337cf16358a40dd` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:46.146929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8573e7c4d2d4404b9337cf16358a40dd)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:46.151749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:46.318114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8573e7c4d2d4404b9337cf16358a40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:46.383894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8573e7c4d2d4404b9337cf16358a40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:46.385148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8573e7c4d2d4404b9337cf16358a40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:46.406049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8573e7c4d2d4404b9337cf16358a40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:46.406819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8573e7c4d2d4404b9337cf16358a40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:46.469733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 948/1899] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:45 2022] DELETE /identity/v3/users/8573e7c4d2d4404b9337cf16358a40dd => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:47.008095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:47.009670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:47.034917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ec87c-d173-4ec3-aecb-77358f4feed2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:47.119688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:47.121812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:47.134036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:47.135036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:47.135386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:47.135857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] PATH_INFO: `/v3/projects/b2d308e66c6947dda9e49b2196a5011a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:47.141792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d308e66c6947dda9e49b2196a5011a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:47.145680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:47.165426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3f1ef-5f39-4074-8d64-4d4e6883638f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:47.198792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:47.200514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:47.201602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:47.223118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] PATH_INFO: `/v3/projects/cb1f414294364c06be631ba97eead63d` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:47.224442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1f414294364c06be631ba97eead63d)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:47.229964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:47.277327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d308e66c6947dda9e49b2196a5011a', 'request_id': 'req-95c0ae01-8072-4441-8ea9-16079a4c19aa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.286024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d308e66c6947dda9e49b2196a5011a', 'request_id': 'req-95c0ae01-8072-4441-8ea9-16079a4c19aa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.289010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:47.302710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d308e66c6947dda9e49b2196a5011a', 'request_id': 'req-95c0ae01-8072-4441-8ea9-16079a4c19aa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.342614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] The token cache is being invalidate because project b2d308e66c6947dda9e49b2196a5011a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:47.343932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.344763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.347441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 952/1900] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:47 2022] DELETE /identity/v3/projects/b2d308e66c6947dda9e49b2196a5011a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:47.354445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:47.356244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:47.365748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1f414294364c06be631ba97eead63d', 'request_id': 'req-e003f363-9098-419c-95be-844a0bb74925'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.380326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1f414294364c06be631ba97eead63d', 'request_id': 'req-e003f363-9098-419c-95be-844a0bb74925'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.404564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1f414294364c06be631ba97eead63d', 'request_id': 'req-e003f363-9098-419c-95be-844a0bb74925'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.426939 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:47.437381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] The token cache is being invalidate because project cb1f414294364c06be631ba97eead63d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:47.438260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.440485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e003f363-9098-419c-95be-844a0bb74925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.446234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 949/1901] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:47 2022] DELETE /identity/v3/projects/cb1f414294364c06be631ba97eead63d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:47.455839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e343-1322-4d6b-8449-d24ef7a0ed0a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:47.456510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e343-1322-4d6b-8449-d24ef7a0ed0a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:47.457228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e343-1322-4d6b-8449-d24ef7a0ed0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:47.491769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0ae01-8072-4441-8ea9-16079a4c19aa admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:47.497811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4b77e343-1322-4d6b-8449-d24ef7a0ed0a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:47.499359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4b77e343-1322-4d6b-8449-d24ef7a0ed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:47.615501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:47.617479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:47.617873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:47.618280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] PATH_INFO: `/v3/users/0bb3c784213a41c0a374b5746bb82409` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:47.620032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb3c784213a41c0a374b5746bb82409)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:47.620367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 950/1902] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:47.624715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:47.631100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ed38-d2da-4e2c-8b8e-c13ebffd2ad9 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:47.632177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ed38-d2da-4e2c-8b8e-c13ebffd2ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:47.632591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ed38-d2da-4e2c-8b8e-c13ebffd2ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:47.659345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7930ed38-d2da-4e2c-8b8e-c13ebffd2ad9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:47.660475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7930ed38-d2da-4e2c-8b8e-c13ebffd2ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:47.678813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 951/1903] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:47.679301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ed38-d2da-4e2c-8b8e-c13ebffd2ad9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:47.704344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ed38-d2da-4e2c-8b8e-c13ebffd2ad9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:47.706250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb3c784213a41c0a374b5746bb82409'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.707663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb3c784213a41c0a374b5746bb82409'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.719497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:47.721531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:47.722042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:47.722523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:47.723934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:47.724453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:47.728349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:47.731793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb3c784213a41c0a374b5746bb82409'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.733939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb3c784213a41c0a374b5746bb82409'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.753959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb3c784213a41c0a374b5746bb82409'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:47.758565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 952/1904] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:47.760081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:47.762869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:47.778401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d007245-5694-4648-a7c8-0c98c8c4c35c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:47.781047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 953/1905] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:47 2022] DELETE /identity/v3/users/0bb3c784213a41c0a374b5746bb82409 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:47.905807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:47.906503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:47.906976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:47.907106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:47.908055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:47.910609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:47.987322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 953/1906] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:47 2022] POST /identity/v3/projects => generated 371 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:03:47.993943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:48.036072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40de7ff-a9e0-4c2a-8a4e-334316f759a5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:48.051214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:48.148287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:48.149556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:48.149907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:48.150171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:48.152442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:48.154798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:48.182538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 954/1907] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:47 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:48.189415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:48.216547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05807e50-6f49-4fec-99c1-34e77059c08a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:48.222387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:48.231362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:48.233062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:48.233753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:48.235201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:48.238686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:48.280222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 954/1908] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:48.280928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:48.295550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfd9c9-40b1-44fe-8742-c5e45bf126d6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:48.303236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c4d7c-6d09-49a9-993c-3438a1c2ccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:48.304300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4d7c-6d09-49a9-993c-3438a1c2ccb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:48.305133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4d7c-6d09-49a9-993c-3438a1c2ccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:48.305497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4d7c-6d09-49a9-993c-3438a1c2ccb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:48.306461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c4d7c-6d09-49a9-993c-3438a1c2ccb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:48.309046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c4d7c-6d09-49a9-993c-3438a1c2ccb9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:48.327212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 955/1909] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:03:48.333621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:48.394976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9dbcf-d6b1-46e0-b968-578047e2c158 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:48.422350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:48.423872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:48.424454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:48.424902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] PATH_INFO: `/v3/projects/facb7cc1d95345a49be48ad34872149c/users/53bdde17192a419e9e3a97cfc69c562a/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:48.513049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=facb7cc1d95345a49be48ad34872149c, user_id=53bdde17192a419e9e3a97cfc69c562a, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:48.518527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:48.583868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 955/1910] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:48 2022] PUT /identity/v3/projects/facb7cc1d95345a49be48ad34872149c/users/53bdde17192a419e9e3a97cfc69c562a/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:48.603574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:48.603916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:48.604174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:48.656209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] MFA Rules not processed for user `53bdde17192a419e9e3a97cfc69c562a`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:48.658024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:48.824058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 956/1911] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:48.828917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:48.906707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:48.911755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:48.938520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:49.046363 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eeac5-5561-4903-88b1-47071521d090 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:49.059581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:49.061270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:49.061861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:49.065091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:49.067354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:49.072704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:49.110019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cba16-91a2-4139-a909-dbe50e27634e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:49.119774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:49.121556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:49.122371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:49.123833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] PATH_INFO: `/v3/projects/7e07f31c725e46b0a49c664630df43cf` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:49.130447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e07f31c725e46b0a49c664630df43cf)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:49.135377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:49.152865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 956/1912] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:48 2022] POST /identity/v3/projects => generated 371 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:49.156510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:49.194602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:49.248011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e07f31c725e46b0a49c664630df43cf', 'request_id': 'req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.258735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e07f31c725e46b0a49c664630df43cf', 'request_id': 'req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.270263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e07f31c725e46b0a49c664630df43cf', 'request_id': 'req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.294882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc552-eb62-4a55-8342-78089ca506cc admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:49.299904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] The token cache is being invalidate because project 7e07f31c725e46b0a49c664630df43cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:49.300642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.302054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.311390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 957/1913] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:48 2022] DELETE /identity/v3/projects/7e07f31c725e46b0a49c664630df43cf => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:49.313229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:49.315660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:49.408149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:49.409664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:49.409664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:49.410175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:49.411108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:49.415073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:49.443518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 957/1914] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:49 2022] POST /identity/v3/users => generated 292 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:49.450511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:49.454443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:49.468321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e78d9-c0d5-4833-925b-f2a2e0cd077a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:49.470998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a6090-26af-4288-a0e6-e6152a762d48 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:49.485916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:49.487999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:49.488498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:49.489007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] PATH_INFO: `/v3/users/c158eeff2f834b14b29b45c44ba4989c` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:49.493010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c158eeff2f834b14b29b45c44ba4989c)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:49.497761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:49.498933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:49.501743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:49.502251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:49.504394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:49.505768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:49.511723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:49.535207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 958/1915] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:03:49.543994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:49.580241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2486-a11a-4d29-9770-63094d8388aa admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:49.600349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:49.602693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:49.603277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:49.603844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] PATH_INFO: `/v3/projects/e1bd1abd1734408f9cfb5a6ff45dcaa5/users/1841f35eac194b4a8530322fe0754ed3/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:49.631530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bd1abd1734408f9cfb5a6ff45dcaa5, user_id=1841f35eac194b4a8530322fe0754ed3, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:49.633745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c158eeff2f834b14b29b45c44ba4989c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.635241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c158eeff2f834b14b29b45c44ba4989c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.652459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:49.657522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c158eeff2f834b14b29b45c44ba4989c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.664498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c158eeff2f834b14b29b45c44ba4989c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.696131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 959/1916] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:49 2022] PUT /identity/v3/projects/e1bd1abd1734408f9cfb5a6ff45dcaa5/users/1841f35eac194b4a8530322fe0754ed3/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:49.702577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:49.709850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:49.710754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c158eeff2f834b14b29b45c44ba4989c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:49.825256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 958/1917] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:49 2022] DELETE /identity/v3/users/c158eeff2f834b14b29b45c44ba4989c => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:49.825256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:49.847114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:49.867176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:50.048008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:50.065875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67934924-3fb5-43fd-ab9a-aed7815320f4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:50.247196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c43b9-b6d2-4544-9a9b-dd5edb86e9e4 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:50.266734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:50.271089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:50.271449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:50.273481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] PATH_INFO: `/v3/users/0f7b7594a00f47778d912bea94ba831b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:50.277884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7b7594a00f47778d912bea94ba831b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:50.280351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:50.389645 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7b7594a00f47778d912bea94ba831b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:50.390637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7b7594a00f47778d912bea94ba831b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:50.412567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7b7594a00f47778d912bea94ba831b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:50.414163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7b7594a00f47778d912bea94ba831b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:50.435904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-80286ea3-f1da-4a90-936f-38fa2a815b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7b7594a00f47778d912bea94ba831b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:50.443466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:50.445527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:50.445956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:50.446730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:50.448709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:50.451515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:50.459388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 959/1918] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:49 2022] DELETE /identity/v3/users/0f7b7594a00f47778d912bea94ba831b => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:50.466716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:50.467412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:50.467910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:50.487795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 960/1919] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 786 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:50.489245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:50.493593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:50.496658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:50.499350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:50.531543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed955f-1fc3-408e-9184-790cc16d2d71 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:50.620270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 960/1920] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:50.680999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418faa-2fd5-41d6-b8bd-8b5ac93457a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:50.682664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418faa-2fd5-41d6-b8bd-8b5ac93457a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:50.683144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418faa-2fd5-41d6-b8bd-8b5ac93457a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:50.684802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418faa-2fd5-41d6-b8bd-8b5ac93457a4 admin admin] PATH_INFO: `/v3/projects/e1bd1abd1734408f9cfb5a6ff45dcaa5/users/1841f35eac194b4a8530322fe0754ed3/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:50.687500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:50.689081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:50.690209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56418faa-2fd5-41d6-b8bd-8b5ac93457a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bd1abd1734408f9cfb5a6ff45dcaa5, user_id=1841f35eac194b4a8530322fe0754ed3, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:50.697062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56418faa-2fd5-41d6-b8bd-8b5ac93457a4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:50.734179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 961/1921] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:50 2022] PUT /identity/v3/projects/e1bd1abd1734408f9cfb5a6ff45dcaa5/users/1841f35eac194b4a8530322fe0754ed3/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:50.744333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:50.745348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:50.746140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:50.809853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05949f78-b3ef-4965-93b0-fe945f521f39 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:50.814932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:50.828776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] MFA Rules not processed for user `1841f35eac194b4a8530322fe0754ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:50.832416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:50.937575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:50.938734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:50.938901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:50.939188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:50.939889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:50.995527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 962/1922] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:51.012268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:51.016840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:51.086055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:51.087588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:51.099489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 961/1923] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:51.152354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c727a4-410e-4a5d-b216-cb3cc026cb04 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:51.160471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:51.162258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:51.163655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:51.164484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:51.184138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:51.186086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:51.199619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 963/1924] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:03:51.649071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:51.649071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:51.673573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2adb92-b8f3-4a05-9cda-baf2fa4d7827 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:51.697095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:51.698957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:51.700019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:51.700513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] PATH_INFO: `/v3/projects/85f3a1c4fb1c4006b99fc3c5bdbb81b8` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:51.703766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f3a1c4fb1c4006b99fc3c5bdbb81b8)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:51.710826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:51.879560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f3a1c4fb1c4006b99fc3c5bdbb81b8', 'request_id': 'req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:51.909619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:51.910411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f3a1c4fb1c4006b99fc3c5bdbb81b8', 'request_id': 'req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:51.916755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:51.924106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f3a1c4fb1c4006b99fc3c5bdbb81b8', 'request_id': 'req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:51.984655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfaaf1-4064-4f88-8a3d-2464c4386d70 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:51.988241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] The token cache is being invalidate because project 85f3a1c4fb1c4006b99fc3c5bdbb81b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:51.991356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:51.996654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ac0ecc98-4cbb-4688-95ae-23d0de47ad0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.003485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 962/1925] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:51 2022] DELETE /identity/v3/projects/85f3a1c4fb1c4006b99fc3c5bdbb81b8 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:52.012722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:52.013211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:52.013515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:52.077086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:52.080146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:52.131039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:52.133106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:52.133398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:52.133685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] PATH_INFO: `/v3/projects/e9cdb70897dd483c817348e1e73fd314` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:52.135193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9cdb70897dd483c817348e1e73fd314)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:52.137659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:52.194594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:52.282157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9cdb70897dd483c817348e1e73fd314', 'request_id': 'req-33547495-87f6-4260-a6af-b7623204510d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.295871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9cdb70897dd483c817348e1e73fd314', 'request_id': 'req-33547495-87f6-4260-a6af-b7623204510d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.309133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 963/1926] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:52.309133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:52.309133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:52.318531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9cdb70897dd483c817348e1e73fd314', 'request_id': 'req-33547495-87f6-4260-a6af-b7623204510d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.348089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ea91e-356b-4d05-84bb-7fefb2c82005 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:52.352502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] The token cache is being invalidate because project e9cdb70897dd483c817348e1e73fd314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:52.353529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.355607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-33547495-87f6-4260-a6af-b7623204510d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.363936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 964/1927] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:51 2022] DELETE /identity/v3/projects/e9cdb70897dd483c817348e1e73fd314 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:03:52.363936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:52.364368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:52.364950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:52.401805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:52.403999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:52.424145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:52.465650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:52.467803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:52.468321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:52.468901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] PATH_INFO: `/v3/users/53bdde17192a419e9e3a97cfc69c562a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:52.470988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bdde17192a419e9e3a97cfc69c562a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:52.474707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:52.573283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 965/1928] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:52.575298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:52.578269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:52.596216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bdde17192a419e9e3a97cfc69c562a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.597461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bdde17192a419e9e3a97cfc69c562a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.604721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64809591-1de6-481f-964a-21d056d3eb91 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:52.617971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:52.619882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:52.620601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:52.621482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] PATH_INFO: `/v3/users/11a7966763f54703953a9f902b8de1fb` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:52.624730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a7966763f54703953a9f902b8de1fb)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:52.629722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:52.630872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bdde17192a419e9e3a97cfc69c562a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.632953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bdde17192a419e9e3a97cfc69c562a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.660032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bdde17192a419e9e3a97cfc69c562a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.703032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 964/1929] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:52 2022] DELETE /identity/v3/users/53bdde17192a419e9e3a97cfc69c562a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:52.704039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:52.705662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:52.755445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a7966763f54703953a9f902b8de1fb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.788934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a7966763f54703953a9f902b8de1fb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.790183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a7966763f54703953a9f902b8de1fb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.813524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a7966763f54703953a9f902b8de1fb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.815853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-590925ef-a0f7-422c-8858-10cc52d45716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a7966763f54703953a9f902b8de1fb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:52.856471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 966/1930] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:52 2022] DELETE /identity/v3/users/11a7966763f54703953a9f902b8de1fb => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:52.860418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:52.860418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:52.860418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:52.899730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:52.902296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:52.970388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d92e5-d875-4d89-b96b-ec67769d0bd7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:52.987448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:53.114489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 967/1931] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:53.166277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:53.167306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:53.167306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:53.167639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:53.168467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:53.169070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:53.171510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:53.188695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 965/1932] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:53.194623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:53.197647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:53.229825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb930d-7710-4cd1-bbcf-b90078238d1a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:53.241900 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:53.244135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:53.247996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:53.249732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:53.250665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:53.251448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:53.253894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:53.260876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:53.340009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 968/1933] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:53 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:53.345405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:53.366469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:53.392683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79549159-eb4d-4280-98a4-caaa1f7796a5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:53.565864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:53.566661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:53.567059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:53.567445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:53.640082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93015023-1874-4943-a0da-e3bb2e1714ab admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:53.671303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:53.673052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:53.673052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:53.673380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:53.674617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:53.699961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:53.783678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 969/1934] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:53 2022] POST /identity/v3/users => generated 293 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:03:53.794747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:53.802505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:53.905976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81e1b7-c6c6-431c-bc7c-34befa58ec62 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:53.907169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:53.913718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:53.925641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:53.926566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:53.926854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:53.927134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] PATH_INFO: `/v3/projects/7396c78b63f04a2d96eb909f481c5fd9` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:53.931439 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7396c78b63f04a2d96eb909f481c5fd9)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:53.937340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:53.944154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 966/1935] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:53.946916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:53.983402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e49048-ee05-4a6a-9981-ddc51a2a7e50 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:54.011966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:54.014785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.016130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.017787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.021143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:54.026323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:54.034386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:54.070399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7396c78b63f04a2d96eb909f481c5fd9', 'request_id': 'req-e3e98483-2174-40eb-8fa8-4bdc64b029c1'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:54.093437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 967/1936] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:54.095141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:54.115189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7396c78b63f04a2d96eb909f481c5fd9', 'request_id': 'req-e3e98483-2174-40eb-8fa8-4bdc64b029c1'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:54.132905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7396c78b63f04a2d96eb909f481c5fd9', 'request_id': 'req-e3e98483-2174-40eb-8fa8-4bdc64b029c1'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:54.136123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecfe9c-2ff9-4290-adeb-ab924ff044eb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:54.157229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] The token cache is being invalidate because project 7396c78b63f04a2d96eb909f481c5fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:54.157555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:54.159933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e3e98483-2174-40eb-8fa8-4bdc64b029c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:54.161851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 970/1937] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:53 2022] DELETE /identity/v3/projects/7396c78b63f04a2d96eb909f481c5fd9 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:54.174341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46205e9-17ed-4f68-9054-047b72708282 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.175217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46205e9-17ed-4f68-9054-047b72708282 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.176645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46205e9-17ed-4f68-9054-047b72708282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.216476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e46205e9-17ed-4f68-9054-047b72708282 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:54.219471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e46205e9-17ed-4f68-9054-047b72708282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:54.309611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:54.311116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.311924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.312520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.313738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:54.317771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:54.355337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 968/1938] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:54.357310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:54.360225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:54.388396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 971/1939] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:54.392128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6342078-79db-46f8-8a4e-b74f63aa88cb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:54.394862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc1f4d-c4eb-4d05-b133-d11df70816d6 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.395837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc1f4d-c4eb-4d05-b133-d11df70816d6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.396614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc1f4d-c4eb-4d05-b133-d11df70816d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.405424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16936ed9-cd99-4f8e-b19e-0a2eae65fa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:54.408178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936ed9-cd99-4f8e-b19e-0a2eae65fa98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.408757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936ed9-cd99-4f8e-b19e-0a2eae65fa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.409479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936ed9-cd99-4f8e-b19e-0a2eae65fa98 admin admin] PATH_INFO: `/v3/projects/6c008b921fd845fba127bcea93adb0ef/users/047f2422197e4d208e2264a8e05a83dd/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.442139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-91bc1f4d-c4eb-4d05-b133-d11df70816d6 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:54.445590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-91bc1f4d-c4eb-4d05-b133-d11df70816d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:54.471967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 972/1940] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:54.474766 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc1f4d-c4eb-4d05-b133-d11df70816d6 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:54.504967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc1f4d-c4eb-4d05-b133-d11df70816d6 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:54.506051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16936ed9-cd99-4f8e-b19e-0a2eae65fa98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c008b921fd845fba127bcea93adb0ef, user_id=047f2422197e4d208e2264a8e05a83dd, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:54.514253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16936ed9-cd99-4f8e-b19e-0a2eae65fa98 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:54.520104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:54.521405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.523738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.524327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.525670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:54.526204 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:54.528896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:54.543914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:54.557594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 973/1941] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:54.571223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 969/1942] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:54 2022] PUT /identity/v3/projects/6c008b921fd845fba127bcea93adb0ef/users/047f2422197e4d208e2264a8e05a83dd/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:54.572571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:54.581997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.582503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.582980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.604381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df66c7-0da8-425f-a34f-8d45d497e7d5 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:54.661833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] MFA Rules not processed for user `047f2422197e4d208e2264a8e05a83dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:54.665140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:54.840346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:54.842484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:54.843319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:54.844525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:54.852569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:54.863802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:54.969575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 970/1943] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:54.982549 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 974/1944] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:54 2022] POST /identity/v3/projects => generated 383 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:54.989367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:54.991652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:54.998892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:55.181799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d1e98-c952-4f47-87b0-2df23a9de35a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:55.202208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab17a4-6379-4ed0-92c4-a848d5d29efb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:55.207835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:55.208487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:55.210831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:55.211368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:55.211805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:55.212864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:55.215112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:55.257081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 975/1945] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:54 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:55.263465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:55.290069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f65dc-dd9f-4e59-9aad-7c7f8519e740 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:55.309975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:55.311816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:55.312441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:55.313104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:55.314624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4eea5-75eb-41a7-882e-f88070d4f3ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:55.315017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:55.316658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4eea5-75eb-41a7-882e-f88070d4f3ea service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:55.317613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4eea5-75eb-41a7-882e-f88070d4f3ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:55.318514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4eea5-75eb-41a7-882e-f88070d4f3ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:55.318875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:55.342282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 976/1946] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:03:55.348883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:55.374664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2e734-fd7d-496a-a7b7-12860e1c2bb8 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:55.387802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:55.389237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:55.389880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:55.390494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:55.392503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:55.396585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:55.421891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4eea5-75eb-41a7-882e-f88070d4f3ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:55.423744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4eea5-75eb-41a7-882e-f88070d4f3ea service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:55.430797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 977/1947] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:03:55.436690 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:55.462218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 971/1948] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:55.466443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e35a7-f0c6-4fa0-b3ba-9038ba215952 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:55.485392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:55.488452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:55.491935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:55.491935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] PATH_INFO: `/v3/projects/58753e247b024c548631d6cbdb8db85f/users/5fe78a0b83934d98b3b17aa50cb74c1a/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:55.549874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58753e247b024c548631d6cbdb8db85f, user_id=5fe78a0b83934d98b3b17aa50cb74c1a, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:55.559348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:55.583511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 978/1949] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:03:55 2022] PUT /identity/v3/projects/58753e247b024c548631d6cbdb8db85f/users/5fe78a0b83934d98b3b17aa50cb74c1a/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:55.603478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e2d5-be32-4229-b150-7f6ea5ac749c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:55.604649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e2d5-be32-4229-b150-7f6ea5ac749c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:55.605186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e2d5-be32-4229-b150-7f6ea5ac749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:55.661170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0bf2e2d5-be32-4229-b150-7f6ea5ac749c None None] MFA Rules not processed for user `5fe78a0b83934d98b3b17aa50cb74c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:55.664091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0bf2e2d5-be32-4229-b150-7f6ea5ac749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:55.832361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 972/1950] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:55.891251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:55.895207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:55.967204 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:56.030919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81e301-1b32-4984-bce8-b6d0aaaa44b3 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:56.039200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:56.045497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:56.047654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:56.048308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:56.075744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:56.080449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:56.107163 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 979/1951] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:56.164692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2e2d5-be32-4229-b150-7f6ea5ac749c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:56.531063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:56.554350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2e2d5-be32-4229-b150-7f6ea5ac749c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:56.569949 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:56.578720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:56.579568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:56.580346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] PATH_INFO: `/v3/users/5fe78a0b83934d98b3b17aa50cb74c1a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:56.583098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe78a0b83934d98b3b17aa50cb74c1a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:56.588959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:56.734255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:56.738731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:56.757370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe78a0b83934d98b3b17aa50cb74c1a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:56.761204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe78a0b83934d98b3b17aa50cb74c1a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:56.790732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0ac68-91c4-4050-a0f2-4aa3743ea31f service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:56.795746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe78a0b83934d98b3b17aa50cb74c1a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:56.798795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe78a0b83934d98b3b17aa50cb74c1a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:56.805097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:56.808380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:56.809607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:56.810302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] PATH_INFO: `/v3/projects/facb7cc1d95345a49be48ad34872149c` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:56.813325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=facb7cc1d95345a49be48ad34872149c)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:56.817791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:56.862308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fbf4c8a2-586d-4061-9c89-205c9640a708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe78a0b83934d98b3b17aa50cb74c1a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:56.918102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 973/1952] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:56 2022] DELETE /identity/v3/users/5fe78a0b83934d98b3b17aa50cb74c1a => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:56.925950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:56.926291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:56.926586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:56.979449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:56.983429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:56.986673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'facb7cc1d95345a49be48ad34872149c', 'request_id': 'req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.002453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'facb7cc1d95345a49be48ad34872149c', 'request_id': 'req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.021232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'facb7cc1d95345a49be48ad34872149c', 'request_id': 'req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.043404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] The token cache is being invalidate because project facb7cc1d95345a49be48ad34872149c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:57.044181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.045302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.048524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 980/1953] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:56 2022] DELETE /identity/v3/projects/facb7cc1d95345a49be48ad34872149c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:57.055451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:57.057697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:57.111803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 974/1954] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:57.132342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:57.134876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:57.294240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a311a5-1f96-4cd7-90dc-0bef735ccb0a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:57.309057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:57.310277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:57.311727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:57.312301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:57.313924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db3ff2-3f4f-4957-815c-20a4bb3fc660 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:57.342685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:57.345151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:57.346051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:57.352742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:57.352742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:57.355985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] PATH_INFO: `/v3/users/1841f35eac194b4a8530322fe0754ed3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:57.358183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1841f35eac194b4a8530322fe0754ed3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:57.364093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:57.431275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 975/1955] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:03:57.536760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1841f35eac194b4a8530322fe0754ed3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.552518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:57.574124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1841f35eac194b4a8530322fe0754ed3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.575105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1841f35eac194b4a8530322fe0754ed3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.590051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1841f35eac194b4a8530322fe0754ed3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.591865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1841f35eac194b4a8530322fe0754ed3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:57.626551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 981/1956] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:03:57 2022] DELETE /identity/v3/users/1841f35eac194b4a8530322fe0754ed3 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:58.685235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:58.698189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db691b-f9f8-4de7-8ea6-7b1479c11748 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:58.719631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:58.767395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:58.768935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:58.832533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:58.833530 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:58.835295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:58.835592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] PATH_INFO: `/v3/projects/e1bd1abd1734408f9cfb5a6ff45dcaa5` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:58.837237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bd1abd1734408f9cfb5a6ff45dcaa5)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:58.841641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:58.944889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bd1abd1734408f9cfb5a6ff45dcaa5', 'request_id': 'req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:58.949848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9025c5-c85a-4668-8747-1e70f09f2703 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:58.954219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bd1abd1734408f9cfb5a6ff45dcaa5', 'request_id': 'req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:58.966008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bd1abd1734408f9cfb5a6ff45dcaa5', 'request_id': 'req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.024555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] The token cache is being invalidate because project e1bd1abd1734408f9cfb5a6ff45dcaa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:59.025134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.028129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c8bd5e20-ac69-47ac-94ba-c7eee93b1e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.031625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 976/1957] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:58 2022] DELETE /identity/v3/projects/e1bd1abd1734408f9cfb5a6ff45dcaa5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:59.038678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5f89d-3a37-4313-811f-dbd606da53f1 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.039438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5f89d-3a37-4313-811f-dbd606da53f1 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.041454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5f89d-3a37-4313-811f-dbd606da53f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.093624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-67a5f89d-3a37-4313-811f-dbd606da53f1 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:59.095446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-67a5f89d-3a37-4313-811f-dbd606da53f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:59.114917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:59.116220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.116569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.117041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] PATH_INFO: `/v3/projects/58753e247b024c548631d6cbdb8db85f` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.119252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58753e247b024c548631d6cbdb8db85f)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:59.122157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:59.199892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58753e247b024c548631d6cbdb8db85f', 'request_id': 'req-921b2879-ea0b-4c18-9837-aa36027b7a76'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.207508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58753e247b024c548631d6cbdb8db85f', 'request_id': 'req-921b2879-ea0b-4c18-9837-aa36027b7a76'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.224285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58753e247b024c548631d6cbdb8db85f', 'request_id': 'req-921b2879-ea0b-4c18-9837-aa36027b7a76'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.246761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] The token cache is being invalidate because project 58753e247b024c548631d6cbdb8db85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:03:59.247060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.248235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-921b2879-ea0b-4c18-9837-aa36027b7a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:03:59.257638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 982/1958] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:03:58 2022] DELETE /identity/v3/projects/58753e247b024c548631d6cbdb8db85f => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:03:59.258988 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6d345-a9af-4b92-ba5e-0a2b6ac9750d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.261278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6d345-a9af-4b92-ba5e-0a2b6ac9750d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.261563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6d345-a9af-4b92-ba5e-0a2b6ac9750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.264742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 977/1959] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:59.275317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47a870-437b-4985-942c-c4cce5654c11 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.275612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47a870-437b-4985-942c-c4cce5654c11 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.275893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47a870-437b-4985-942c-c4cce5654c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.298321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-dc47a870-437b-4985-942c-c4cce5654c11 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:59.299587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dc47a870-437b-4985-942c-c4cce5654c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:59.301309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-22a6d345-a9af-4b92-ba5e-0a2b6ac9750d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:59.303679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-22a6d345-a9af-4b92-ba5e-0a2b6ac9750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:59.323083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:59.365062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:03:59.391324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 978/1960] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:03:59.396177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47a870-437b-4985-942c-c4cce5654c11 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:59.408669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47a870-437b-4985-942c-c4cce5654c11 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:59.415523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:59.416260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.416566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.416848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.418282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:59.418687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:59.423560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:59.458824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 983/1961] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:59.463077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a0019-66b5-495f-b5c0-56830f6bc684 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.464073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a0019-66b5-495f-b5c0-56830f6bc684 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.464885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 979/1962] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:59.465072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a0019-66b5-495f-b5c0-56830f6bc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.466758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:59.503210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc36f-d7c8-4d52-9789-9801a256601f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:59.513672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:59.515003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.515730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.516769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.517486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5a5a0019-66b5-495f-b5c0-56830f6bc684 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:59.517806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:59.519438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5a5a0019-66b5-495f-b5c0-56830f6bc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:59.524470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:59.531007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 984/1963] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:59.536468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a0019-66b5-495f-b5c0-56830f6bc684 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:59.558886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a0019-66b5-495f-b5c0-56830f6bc684 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:59.567800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:59.567800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.568123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.568793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.569761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:03:59.570075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:59.572777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:59.607191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 980/1964] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/projects => generated 355 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:59.611009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 985/1965] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:03:59.611231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:59.624412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:59.637029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff35267-9d84-4425-9597-b275c14af8a1 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:59.746134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a888b8d-4a4f-4961-878d-d02c2676c2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:59.747105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b8d-4a4f-4961-878d-d02c2676c2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.747589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b8d-4a4f-4961-878d-d02c2676c2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.748423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b8d-4a4f-4961-878d-d02c2676c2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.752265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f10136-7c28-467c-a28c-7e7e96591c6b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:59.753141 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b8d-4a4f-4961-878d-d02c2676c2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:59.755838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b8d-4a4f-4961-878d-d02c2676c2e2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:59.764102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c8610-0850-4fc2-928c-bfb5acdcc9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:59.766035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c8610-0850-4fc2-928c-bfb5acdcc9fb admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.766959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c8610-0850-4fc2-928c-bfb5acdcc9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.767828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c8610-0850-4fc2-928c-bfb5acdcc9fb admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.770132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c8610-0850-4fc2-928c-bfb5acdcc9fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:03:59.776236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c8610-0850-4fc2-928c-bfb5acdcc9fb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:03:59.802176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 986/1966] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:03:59.804269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84c378-f5d0-43cc-a3af-b88364335d75 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.804614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84c378-f5d0-43cc-a3af-b88364335d75 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.805119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84c378-f5d0-43cc-a3af-b88364335d75 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.806875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 987/1967] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:03:59.808005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84c378-f5d0-43cc-a3af-b88364335d75 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:59.822782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 981/1968] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/users => generated 278 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:03:59.826240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea324-7db4-463d-a9f8-b22811c35aa4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.827605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea324-7db4-463d-a9f8-b22811c35aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.828475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea324-7db4-463d-a9f8-b22811c35aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.874765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f3cea324-7db4-463d-a9f8-b22811c35aa4 None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:03:59.876332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f3cea324-7db4-463d-a9f8-b22811c35aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:03:59.964375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84c378-f5d0-43cc-a3af-b88364335d75 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:03:59.978396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:03:59.981030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:03:59.981882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:03:59.983581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:03:59.984921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 982/1969] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:03:59.988176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cea324-7db4-463d-a9f8-b22811c35aa4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:03:59.998058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.002703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.003060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cea324-7db4-463d-a9f8-b22811c35aa4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.019101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.019816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.020107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.020390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.021183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.023020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.048344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 988/1970] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:03:59 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:00.052417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.056234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 983/1971] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:00.057068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.060085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:00.073840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ed839-7740-4adb-9d45-6765e8ff4d45 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.080313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.081231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.081694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.082090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.082974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.085886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.090455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:00.098808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96c13e-7714-45ca-8c31-0e8c6b35fc68 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.108673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 984/1972] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:00.110593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.115027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.116306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.116773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.117095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.129251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.131757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.132830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d43ff-dd0f-4001-8023-6535d6c84603 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.141850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.143038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.143540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.144032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.145456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.149480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 989/1973] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:00.149701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.151981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.169460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 985/1974] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:00.173217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f1e8-88f2-4703-99a6-2f37e3f475db service placement] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.177569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.185138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.186060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.186410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.186751 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.187583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.196037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.208702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464c033-8000-4f79-b2fe-3bc29d25a34c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.209046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 990/1975] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:00.214914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.225391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d0c2c-6a8e-46cc-b884-c198c62ef34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.226582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d0c2c-6a8e-46cc-b884-c198c62ef34a admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.226823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d0c2c-6a8e-46cc-b884-c198c62ef34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.227173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d0c2c-6a8e-46cc-b884-c198c62ef34a admin admin] PATH_INFO: `/v3/projects/473bb304cb3b4dedab80f534d490e2ef/users/30b4b62251104fc98d638c0b8561b602/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.231910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:00.234913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb0dc9-3006-436a-90f4-70c32c2eec8a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.243895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1a40e-cdba-4857-b768-25e5f5690d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.246284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a40e-cdba-4857-b768-25e5f5690d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.246709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a40e-cdba-4857-b768-25e5f5690d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.247297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a40e-cdba-4857-b768-25e5f5690d8f admin admin] PATH_INFO: `/v3/projects/ef4bdc1e6e07411fac23ea832d862204/users/52837a163874423785ca6fcda7765d57/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.250892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d0c2c-6a8e-46cc-b884-c198c62ef34a admin admin] RBAC: Authorizing `identity:create_grant(project_id=473bb304cb3b4dedab80f534d490e2ef, user_id=30b4b62251104fc98d638c0b8561b602, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.254026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d0c2c-6a8e-46cc-b884-c198c62ef34a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.268959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1a40e-cdba-4857-b768-25e5f5690d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4bdc1e6e07411fac23ea832d862204, user_id=52837a163874423785ca6fcda7765d57, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.272758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1a40e-cdba-4857-b768-25e5f5690d8f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.289207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 986/1976] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:00 2022] PUT /identity/v3/projects/473bb304cb3b4dedab80f534d490e2ef/users/30b4b62251104fc98d638c0b8561b602/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:00.301035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 991/1977] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:00 2022] PUT /identity/v3/projects/ef4bdc1e6e07411fac23ea832d862204/users/52837a163874423785ca6fcda7765d57/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:00.308490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5754bb-4cb9-456f-a874-ab58737110d0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.309131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5754bb-4cb9-456f-a874-ab58737110d0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.309520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5754bb-4cb9-456f-a874-ab58737110d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.312855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.313341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.313741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.359346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1b5754bb-4cb9-456f-a874-ab58737110d0 None None] MFA Rules not processed for user `30b4b62251104fc98d638c0b8561b602`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:00.363861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] MFA Rules not processed for user `52837a163874423785ca6fcda7765d57`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:00.364620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1b5754bb-4cb9-456f-a874-ab58737110d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:00.366042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:00.501732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 987/1978] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:00 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:00.508140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5754bb-4cb9-456f-a874-ab58737110d0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.510018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 992/1979] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:00.546983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.553618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:00.667847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5754bb-4cb9-456f-a874-ab58737110d0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.679343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49102d55-0912-452d-aefa-db90abb8537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.680942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49102d55-0912-452d-aefa-db90abb8537a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.681572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49102d55-0912-452d-aefa-db90abb8537a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.683934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49102d55-0912-452d-aefa-db90abb8537a admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.685520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49102d55-0912-452d-aefa-db90abb8537a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.691322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49102d55-0912-452d-aefa-db90abb8537a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.692432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ebe4a-c730-4f59-aa28-4797e3c8b67e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.700970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:00.702310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.703010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.703531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.724387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:00.728729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:00.752521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 993/1980] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:00.769267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 988/1981] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:00 2022] POST /identity/v3/projects => generated 357 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:00.774538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:00.795053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cfa2f-39d7-4735-a068-ca411066b7ad service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:00.817479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f68f6b-688c-47b5-baa6-fdf8d7e4d15d None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.818045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f68f6b-688c-47b5-baa6-fdf8d7e4d15d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.818806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f68f6b-688c-47b5-baa6-fdf8d7e4d15d None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.821803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 989/1982] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:04:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:04:00.828684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f693c-5bce-4d25-a665-6cf7e5c52029 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:00.829263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f693c-5bce-4d25-a665-6cf7e5c52029 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:00.829773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f693c-5bce-4d25-a665-6cf7e5c52029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:00.891225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6c6f693c-5bce-4d25-a665-6cf7e5c52029 None None] MFA Rules not processed for user `4fdc728b26894afa85073c08c7c34ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:00.893991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6c6f693c-5bce-4d25-a665-6cf7e5c52029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:00.976942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:01.011591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:01.013020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.013618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.014457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.015535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:01.017852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:01.038525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:01.065676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 994/1983] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:00 2022] POST /identity/v3/users => generated 285 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:01.067366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:01.075330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:01.094316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da27f-0c2e-4dda-bb7a-465a9446973e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:01.097730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 990/1984] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:04:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:01.099078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f693c-5bce-4d25-a665-6cf7e5c52029 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:01.100090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:01.101035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.101353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.101799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] PATH_INFO: `/v3/users/2c94b4b5a48a4a2bbbac9d1e916e09dd` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.103143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c94b4b5a48a4a2bbbac9d1e916e09dd)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:01.105199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:01.126360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f693c-5bce-4d25-a665-6cf7e5c52029 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:01.138660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:01.139374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.139685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.140051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.140707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:01.143992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:01.159181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 991/1985] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:01.166163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:01.181127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698c6d-1c70-4b73-b4c2-2adbfe147eb2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:01.198732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:01.199855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.200286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.200816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] PATH_INFO: `/v3/projects/517e03f54eff47039a3e52ae030c94bc/users/91f2c92618fd429db76f2288e8b63edd/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.223637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517e03f54eff47039a3e52ae030c94bc, user_id=91f2c92618fd429db76f2288e8b63edd, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:01.231809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:01.238941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c94b4b5a48a4a2bbbac9d1e916e09dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:01.265212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 992/1986] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:01 2022] PUT /identity/v3/projects/517e03f54eff47039a3e52ae030c94bc/users/91f2c92618fd429db76f2288e8b63edd/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:01.268132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:01.290917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c94b4b5a48a4a2bbbac9d1e916e09dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:01.292002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c94b4b5a48a4a2bbbac9d1e916e09dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:01.315214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c94b4b5a48a4a2bbbac9d1e916e09dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:01.316640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c94b4b5a48a4a2bbbac9d1e916e09dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:01.352750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 995/1987] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:01 2022] DELETE /identity/v3/users/2c94b4b5a48a4a2bbbac9d1e916e09dd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:01.354121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:01.359224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:01.533600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c370f3-1f48-45d6-ad55-19d4cb0d8c5c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:01.543393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:01.544407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.544860 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.545411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.547698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:01.550707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4aeb3-2c37-44a0-8b4c-cff7be609a43 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:01.555174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:01.612543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 996/1988] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:01.614938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:01.655044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9840c9-ed9c-4ed1-bd28-bd039848a251 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:01.666199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53eee7-e892-43b4-a32e-5f84927213ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:01.667802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53eee7-e892-43b4-a32e-5f84927213ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.669019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53eee7-e892-43b4-a32e-5f84927213ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.669526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53eee7-e892-43b4-a32e-5f84927213ec admin admin] PATH_INFO: `/v3/projects/517e03f54eff47039a3e52ae030c94bc/users/91f2c92618fd429db76f2288e8b63edd/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.675351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53eee7-e892-43b4-a32e-5f84927213ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=517e03f54eff47039a3e52ae030c94bc, user_id=91f2c92618fd429db76f2288e8b63edd, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:01.681886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53eee7-e892-43b4-a32e-5f84927213ec admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:01.686842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:01.688058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.688518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.689131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.698322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:01.725929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:01.729386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:01.733021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 997/1989] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:01 2022] PUT /identity/v3/projects/517e03f54eff47039a3e52ae030c94bc/users/91f2c92618fd429db76f2288e8b63edd/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:01.748423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:01.748619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:01.748875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:01.752525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 993/1990] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:01.801438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] MFA Rules not processed for user `91f2c92618fd429db76f2288e8b63edd`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:01.803384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:01.841096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:01.917153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 998/1991] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:01 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:01.936849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:01.939631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:02.065718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45b10d-32b8-4948-a12b-0a2fce8926bd service placement] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:02.073268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:02.165445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:02.166271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:02.166783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:02.167130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:02.187803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:02.190179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:02.193125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:02.196289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:02.205760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 994/1992] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:01 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:02.420308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9acbc-63fe-4a33-a40e-8708985f73d4 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:02.433277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:02.435039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:02.435474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:02.435912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] PATH_INFO: `/v3/projects/4f33904a6b474414bd4632b2b024025c` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:02.438289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f33904a6b474414bd4632b2b024025c)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:02.442310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:02.581968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f33904a6b474414bd4632b2b024025c', 'request_id': 'req-9765541e-2b7d-446b-a3c6-67ad300bc8ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.595457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f33904a6b474414bd4632b2b024025c', 'request_id': 'req-9765541e-2b7d-446b-a3c6-67ad300bc8ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.620669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f33904a6b474414bd4632b2b024025c', 'request_id': 'req-9765541e-2b7d-446b-a3c6-67ad300bc8ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.648880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:02.651066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] The token cache is being invalidate because project 4f33904a6b474414bd4632b2b024025c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:02.651593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.653008 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.656623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 999/1993] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:02 2022] DELETE /identity/v3/projects/4f33904a6b474414bd4632b2b024025c => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:02.664101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:02.666708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:02.672920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cda996-a5bd-4959-bc55-8eeae1cc8749 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:02.808024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:02.821636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:02.823095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:02.823095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:02.823095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:02.835030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:02.841356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765541e-2b7d-446b-a3c6-67ad300bc8ae admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:02.848218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:02.849491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:02.851263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:02.852226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] PATH_INFO: `/v3/users/d7632ef499014cde8c7f2610040b521e` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:02.854078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7632ef499014cde8c7f2610040b521e)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:02.856101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:02.950347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:02.952185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7632ef499014cde8c7f2610040b521e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.953616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:02.968309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 995/1994] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:02 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:02.982830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7632ef499014cde8c7f2610040b521e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.984454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7632ef499014cde8c7f2610040b521e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:02.999082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7632ef499014cde8c7f2610040b521e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:03.000022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-94c5bf5a-caad-4e4a-afcb-79363a190190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7632ef499014cde8c7f2610040b521e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:03.023705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1000/1995] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:02 2022] DELETE /identity/v3/users/d7632ef499014cde8c7f2610040b521e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:03.380366 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:03.405382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eac1a-d989-4c8c-956d-aff4cbfac9bd service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:03.487649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:03.488511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:03.488819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:03.489179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] PATH_INFO: `/v3/projects/bb208e395a2e4fb7b16a7e21dba1e573` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:03.491836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb208e395a2e4fb7b16a7e21dba1e573)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:03.493663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:03.507938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:03.568089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb208e395a2e4fb7b16a7e21dba1e573', 'request_id': 'req-595ab5af-74d8-4ae1-a21b-65109b5c915d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:03.575093 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb208e395a2e4fb7b16a7e21dba1e573', 'request_id': 'req-595ab5af-74d8-4ae1-a21b-65109b5c915d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:03.582117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb208e395a2e4fb7b16a7e21dba1e573', 'request_id': 'req-595ab5af-74d8-4ae1-a21b-65109b5c915d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:03.596940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] The token cache is being invalidate because project bb208e395a2e4fb7b16a7e21dba1e573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:03.597260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:03.597939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-595ab5af-74d8-4ae1-a21b-65109b5c915d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:03.599556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 996/1996] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:03 2022] DELETE /identity/v3/projects/bb208e395a2e4fb7b16a7e21dba1e573 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:03.605118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e019-bee5-43ef-905b-fc2c1810cd9e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:03.605645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e019-bee5-43ef-905b-fc2c1810cd9e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:03.605914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e019-bee5-43ef-905b-fc2c1810cd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:03.625581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4031e019-bee5-43ef-905b-fc2c1810cd9e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:03.626825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4031e019-bee5-43ef-905b-fc2c1810cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:03.696170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1001/1997] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:03.705938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9200-7f5b-48a3-be4f-a00ab09f51b4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:03.706527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9200-7f5b-48a3-be4f-a00ab09f51b4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:03.707121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9200-7f5b-48a3-be4f-a00ab09f51b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:03.744788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-47ae9200-7f5b-48a3-be4f-a00ab09f51b4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:03.747403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-47ae9200-7f5b-48a3-be4f-a00ab09f51b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:03.761242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 997/1998] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:03.766807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031e019-bee5-43ef-905b-fc2c1810cd9e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:03.779676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031e019-bee5-43ef-905b-fc2c1810cd9e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:03.786408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:03.787308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:03.787718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:03.788079 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:03.788895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:03.789287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:03.791504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:03.810647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1002/1999] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:03.816040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae9200-7f5b-48a3-be4f-a00ab09f51b4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:03.828177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae9200-7f5b-48a3-be4f-a00ab09f51b4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:03.833470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:03.834744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:03.835024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:03.835305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:03.836036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:03.838150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:03.877114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 998/2000] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:03 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:03.878507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:03.891147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb30397-249a-4f6d-a2b8-56076d551ffa admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:03.957307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:03.970024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:03.971471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:03.972593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:03.973386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:03.975591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:03.980508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:04.015874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1003/2001] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:03 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:04.021551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:04.036965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f738064-0f48-4eea-b3a1-7cacea79bc5a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:04.047167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:04.048431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.049014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.049519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.050804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:04.053458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:04.065553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 999/2002] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:04.070987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:04.082497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b8fb2-36f0-44e9-ab1c-7e3d8f1db687 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:04.088471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552472e-0153-4c2c-bf94-d1b4321c6b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:04.089263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552472e-0153-4c2c-bf94-d1b4321c6b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.089586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552472e-0153-4c2c-bf94-d1b4321c6b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.089976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552472e-0153-4c2c-bf94-d1b4321c6b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.090804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552472e-0153-4c2c-bf94-d1b4321c6b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:04.092713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552472e-0153-4c2c-bf94-d1b4321c6b35 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:04.106657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1004/2003] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:04.111432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:04.123523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f64d8e-bef6-462b-bb42-1ec95bdbf5ec admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:04.128508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:04.129533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.129831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.130122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] PATH_INFO: `/v3/projects/f05fac638c8146bd978a1467d2403b1a/users/603e341bfa38452c8e4b38fc4bccada9/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.150816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f05fac638c8146bd978a1467d2403b1a, user_id=603e341bfa38452c8e4b38fc4bccada9, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:04.153787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:04.171259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1000/2004] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:04 2022] PUT /identity/v3/projects/f05fac638c8146bd978a1467d2403b1a/users/603e341bfa38452c8e4b38fc4bccada9/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:04.182407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae731593-1892-4758-ac44-36c72549a39b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.182727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae731593-1892-4758-ac44-36c72549a39b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.183018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae731593-1892-4758-ac44-36c72549a39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.213920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ae731593-1892-4758-ac44-36c72549a39b None None] MFA Rules not processed for user `603e341bfa38452c8e4b38fc4bccada9`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:04.215213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ae731593-1892-4758-ac44-36c72549a39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:04.303795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1005/2005] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:04.332355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:04.335245 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:04.452722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78488-0280-4fab-8bdc-fc60b75ae06f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:04.457038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116959b-a739-4455-a5a6-a065b03e6e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:04.457727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116959b-a739-4455-a5a6-a065b03e6e85 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.458009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116959b-a739-4455-a5a6-a065b03e6e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.458293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116959b-a739-4455-a5a6-a065b03e6e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.468803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116959b-a739-4455-a5a6-a065b03e6e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:04.470331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116959b-a739-4455-a5a6-a065b03e6e85 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:04.483398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1001/2006] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:04.523831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae731593-1892-4758-ac44-36c72549a39b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:04.617656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae731593-1892-4758-ac44-36c72549a39b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:04.625110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:04.625956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.626662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.626796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] PATH_INFO: `/v3/users/603e341bfa38452c8e4b38fc4bccada9` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.628092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603e341bfa38452c8e4b38fc4bccada9)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:04.630349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:04.700765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603e341bfa38452c8e4b38fc4bccada9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:04.724327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603e341bfa38452c8e4b38fc4bccada9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:04.725934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603e341bfa38452c8e4b38fc4bccada9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:04.727894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:04.737220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603e341bfa38452c8e4b38fc4bccada9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:04.738780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603e341bfa38452c8e4b38fc4bccada9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:04.757002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1006/2007] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:04 2022] DELETE /identity/v3/users/603e341bfa38452c8e4b38fc4bccada9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:04.762281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.762630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.763000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.783757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:04.785059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:04.822963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:04.863152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1002/2008] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:04.876149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:04.878574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:04.958966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb2ce3-4551-419c-9ba6-1d3c10402804 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:04.963520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff7600-0496-40d2-a723-74f5c6754b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:04.964274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7600-0496-40d2-a723-74f5c6754b14 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:04.964555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7600-0496-40d2-a723-74f5c6754b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:04.964838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7600-0496-40d2-a723-74f5c6754b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:04.976590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff7600-0496-40d2-a723-74f5c6754b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:04.978117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff7600-0496-40d2-a723-74f5c6754b14 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:04.989210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1007/2009] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:05.375216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:05.377765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:05.394720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce565ab-6582-4913-bc8f-db3b583270cb None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:05.399729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:05.400700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:05.401009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:05.401343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] PATH_INFO: `/v3/projects/f05fac638c8146bd978a1467d2403b1a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:05.402758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f05fac638c8146bd978a1467d2403b1a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:05.404867 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:05.482750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f05fac638c8146bd978a1467d2403b1a', 'request_id': 'req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:05.495260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f05fac638c8146bd978a1467d2403b1a', 'request_id': 'req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:05.503239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f05fac638c8146bd978a1467d2403b1a', 'request_id': 'req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:05.522313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] The token cache is being invalidate because project f05fac638c8146bd978a1467d2403b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:05.522854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:05.523713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0f3f0f0e-f229-4c32-9fd1-839d4feedf9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:05.526414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1003/2010] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:05 2022] DELETE /identity/v3/projects/f05fac638c8146bd978a1467d2403b1a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:05.533727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751cea-4ae1-4bf1-a343-89cddd6ac04c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:05.534138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751cea-4ae1-4bf1-a343-89cddd6ac04c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:05.534521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751cea-4ae1-4bf1-a343-89cddd6ac04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:05.565356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5f751cea-4ae1-4bf1-a343-89cddd6ac04c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:05.567606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5f751cea-4ae1-4bf1-a343-89cddd6ac04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:05.642090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1008/2011] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:05.652067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1ecc-94b4-47a7-9182-36f1298866c2 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:05.652412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1ecc-94b4-47a7-9182-36f1298866c2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:05.652751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1ecc-94b4-47a7-9182-36f1298866c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:05.673602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-228e1ecc-94b4-47a7-9182-36f1298866c2 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:05.674943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-228e1ecc-94b4-47a7-9182-36f1298866c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:05.684938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1004/2012] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:05.692568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f751cea-4ae1-4bf1-a343-89cddd6ac04c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:05.712908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f751cea-4ae1-4bf1-a343-89cddd6ac04c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:05.721532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:05.722747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:05.723260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:05.723763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:05.726460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:05.727067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:05.730968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:05.747712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1009/2013] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:05.752513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e1ecc-94b4-47a7-9182-36f1298866c2 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:05.763798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e1ecc-94b4-47a7-9182-36f1298866c2 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:05.769163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:05.769853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:05.770138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:05.770417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:05.771135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:05.774116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:05.810549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1005/2014] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:05 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:05.815557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:05.827475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b497-8cfb-4e02-9086-97864a75eb38 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:05.913508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:05.914200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:05.914482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:05.914757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:05.915470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:05.917519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:05.940533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1010/2015] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:05 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:05.946281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:05.958458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5c461-5df1-49ee-a1c3-33f5af3e748a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:05.964740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:05.965485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:05.965769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:05.966050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:05.966807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:05.968471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:05.981719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1006/2016] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:05.985053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:05.991039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:05.998810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035771b-f5c2-444b-9aba-ce9038cb09b4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:06.005734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35083dbc-8e5c-43dd-8c52-c90e78eb0dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:06.007323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35083dbc-8e5c-43dd-8c52-c90e78eb0dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:06.007768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35083dbc-8e5c-43dd-8c52-c90e78eb0dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:06.008136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35083dbc-8e5c-43dd-8c52-c90e78eb0dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:06.009016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35083dbc-8e5c-43dd-8c52-c90e78eb0dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:06.010772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35083dbc-8e5c-43dd-8c52-c90e78eb0dc3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:06.022895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1011/2017] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:06.027316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:06.039361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390a61-a3ab-465e-a099-10b8213c0244 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:06.044906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:06.045957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:06.046332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:06.046668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] PATH_INFO: `/v3/projects/de8c46aaa09443f485ac01c23c548554/users/88428b2172c24e13a6e0fbb5e70e738d/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:06.064719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8c46aaa09443f485ac01c23c548554, user_id=88428b2172c24e13a6e0fbb5e70e738d, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:06.067487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:06.083803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1007/2018] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:06 2022] PUT /identity/v3/projects/de8c46aaa09443f485ac01c23c548554/users/88428b2172c24e13a6e0fbb5e70e738d/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:06.096529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:06.097052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:06.097535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:06.145081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] MFA Rules not processed for user `88428b2172c24e13a6e0fbb5e70e738d`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:06.148700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:06.249896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1012/2019] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:06.263423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:06.265253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:06.332090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7c02-59a7-4acd-9baa-c6b705095d4d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:06.336817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933edc8-1ef6-42ca-8cb6-8a85922e6f04 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:06.337567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933edc8-1ef6-42ca-8cb6-8a85922e6f04 service glance] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:06.337870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933edc8-1ef6-42ca-8cb6-8a85922e6f04 service glance] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:06.338153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933edc8-1ef6-42ca-8cb6-8a85922e6f04 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:06.352360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933edc8-1ef6-42ca-8cb6-8a85922e6f04 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:06.355133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933edc8-1ef6-42ca-8cb6-8a85922e6f04 service glance] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:06.362119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:06.371020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1008/2020] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:07.674356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:07.676239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:07.763275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbb88-c6aa-414b-9c2c-39c3cb77eba0 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:07.768177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:07.769112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:07.769409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:07.769693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] PATH_INFO: `/v3/users/047f2422197e4d208e2264a8e05a83dd` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:07.770946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=047f2422197e4d208e2264a8e05a83dd)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:07.774555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:07.833090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047f2422197e4d208e2264a8e05a83dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:07.851484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047f2422197e4d208e2264a8e05a83dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:07.852381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047f2422197e4d208e2264a8e05a83dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:07.863558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047f2422197e4d208e2264a8e05a83dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:07.864326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047f2422197e4d208e2264a8e05a83dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:07.876388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:07.883303 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1013/2021] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:07 2022] DELETE /identity/v3/users/047f2422197e4d208e2264a8e05a83dd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:07.889191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:07.889755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:07.889755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:07.909062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:07.910311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:08.029576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1009/2022] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:08.043548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:08.045231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:08.142038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5870c-97c0-44be-a8ad-f88feb4990bb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:08.145950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6de63-943f-4df7-a15e-d48b0867cb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:08.147317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6de63-943f-4df7-a15e-d48b0867cb75 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:08.147881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6de63-943f-4df7-a15e-d48b0867cb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:08.148331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6de63-943f-4df7-a15e-d48b0867cb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:08.161138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6de63-943f-4df7-a15e-d48b0867cb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:08.167268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6de63-943f-4df7-a15e-d48b0867cb75 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:08.175781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1014/2023] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:08.613819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:08.617395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:08.641413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e7963-9012-48c5-a03a-89d8bd752c69 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:08.651586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:08.652545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:08.652843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:08.653190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] PATH_INFO: `/v3/projects/6c008b921fd845fba127bcea93adb0ef` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:08.667298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c008b921fd845fba127bcea93adb0ef)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:08.669669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:08.758998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c008b921fd845fba127bcea93adb0ef', 'request_id': 'req-41a0313d-b656-4192-9ef9-241dc91b7ec1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:08.767803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c008b921fd845fba127bcea93adb0ef', 'request_id': 'req-41a0313d-b656-4192-9ef9-241dc91b7ec1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:08.777582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c008b921fd845fba127bcea93adb0ef', 'request_id': 'req-41a0313d-b656-4192-9ef9-241dc91b7ec1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:08.809356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] The token cache is being invalidate because project 6c008b921fd845fba127bcea93adb0ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:08.809356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:08.812631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-41a0313d-b656-4192-9ef9-241dc91b7ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:08.814881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1010/2024] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:08 2022] DELETE /identity/v3/projects/6c008b921fd845fba127bcea93adb0ef => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:08.822926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f626c7-e9a2-499e-96f3-ac6a856ad0dd None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:08.823761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f626c7-e9a2-499e-96f3-ac6a856ad0dd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:08.824251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f626c7-e9a2-499e-96f3-ac6a856ad0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:08.856951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f4f626c7-e9a2-499e-96f3-ac6a856ad0dd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:08.859906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f4f626c7-e9a2-499e-96f3-ac6a856ad0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:09.003045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1015/2025] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:09.013206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c1166-5640-44b2-9b84-d9923b44ba9f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.013907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c1166-5640-44b2-9b84-d9923b44ba9f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.013907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c1166-5640-44b2-9b84-d9923b44ba9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.046069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cc5c1166-5640-44b2-9b84-d9923b44ba9f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:09.047328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cc5c1166-5640-44b2-9b84-d9923b44ba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:09.061740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1011/2026] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:09.069590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f626c7-e9a2-499e-96f3-ac6a856ad0dd None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.086678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f626c7-e9a2-499e-96f3-ac6a856ad0dd None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.096119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.097459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.098023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.098590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.100136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:09.100760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.105408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.135134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1016/2027] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:09.142029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c1166-5640-44b2-9b84-d9923b44ba9f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.154681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c1166-5640-44b2-9b84-d9923b44ba9f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.161471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.163362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.163766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.164155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.165014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.167406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.218043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1012/2028] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:09 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:09.226597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.245720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f456931-41e0-4a62-9bd4-980d50c08317 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.320413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.321106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.321331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.321612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.322324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.324312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.348662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1017/2029] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:09 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:09.354062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.366520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46d15b-28d7-4fab-b175-bd69d6044296 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.371659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.372365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.372659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.372945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.373756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.375420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.388639 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1013/2030] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:04:09.395184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.414495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600497-5e3f-4d96-81a9-65b1864c7c58 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.420419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d15e-045a-42b4-8bde-9a94b059b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.421240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d15e-045a-42b4-8bde-9a94b059b784 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.421573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d15e-045a-42b4-8bde-9a94b059b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.421896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d15e-045a-42b4-8bde-9a94b059b784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.422695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069d15e-045a-42b4-8bde-9a94b059b784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.424492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069d15e-045a-42b4-8bde-9a94b059b784 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.437806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1018/2031] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:09.442185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.454455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578419-7c7b-4429-b232-6a8917276ea5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.460624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.461600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.461887 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.462164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] PATH_INFO: `/v3/projects/bdd9e1aba3444c0bb815cb0aa45bc7e2/users/0022cb978523450ebeba5ceaadd2132b/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.480168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdd9e1aba3444c0bb815cb0aa45bc7e2, user_id=0022cb978523450ebeba5ceaadd2132b, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.483263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.501816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1014/2032] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:09 2022] PUT /identity/v3/projects/bdd9e1aba3444c0bb815cb0aa45bc7e2/users/0022cb978523450ebeba5ceaadd2132b/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:09.515138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a85964-2a0c-41b1-8543-aafb24b647a6 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.515656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a85964-2a0c-41b1-8543-aafb24b647a6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.516202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a85964-2a0c-41b1-8543-aafb24b647a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.549599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:09.561327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-09a85964-2a0c-41b1-8543-aafb24b647a6 None None] MFA Rules not processed for user `0022cb978523450ebeba5ceaadd2132b`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:09.563223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-09a85964-2a0c-41b1-8543-aafb24b647a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:09.644113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1019/2033] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:09.649876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.671411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:09.730942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1a06a-0b9a-4f11-8a4f-49faece9fbff admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.736837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.738447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.738796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.739405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.740420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.744493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.778259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1015/2034] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:09 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:09.782932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a85964-2a0c-41b1-8543-aafb24b647a6 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.794453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a85964-2a0c-41b1-8543-aafb24b647a6 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.869446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.870627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.871128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.871635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.872969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.876215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.899180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1020/2035] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:09 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:09.903208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.914995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22967c2d-92b8-4601-b291-c5e85f62e325 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.920213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.920969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.921258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.921582 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.922338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.924065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.937878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1016/2036] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:09.944003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:09.962344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ace9a-05f3-4204-bed7-a82286eb82c7 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:09.970657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6017c4c-44a9-4950-8cf1-ecd31c2b6f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:09.971841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6017c4c-44a9-4950-8cf1-ecd31c2b6f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:09.972362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6017c4c-44a9-4950-8cf1-ecd31c2b6f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:09.972880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6017c4c-44a9-4950-8cf1-ecd31c2b6f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:09.974311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6017c4c-44a9-4950-8cf1-ecd31c2b6f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:09.977999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6017c4c-44a9-4950-8cf1-ecd31c2b6f52 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:09.996147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1021/2037] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:10.001961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:10.014695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b5dc-ced2-42db-8f24-2fc2d1b5a6d2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:10.019761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:10.020693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.021004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.022167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] PATH_INFO: `/v3/projects/271400214841469f853aa49b53528f03/users/395c6561137d4ff19da150744f1b1d9d/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.040775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271400214841469f853aa49b53528f03, user_id=395c6561137d4ff19da150744f1b1d9d, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:10.043804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:10.070697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1017/2038] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:10 2022] PUT /identity/v3/projects/271400214841469f853aa49b53528f03/users/395c6561137d4ff19da150744f1b1d9d/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:10.083635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314433-b284-4b79-baa5-6cb7323aee5b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.084646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314433-b284-4b79-baa5-6cb7323aee5b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.085132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314433-b284-4b79-baa5-6cb7323aee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.116739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c7314433-b284-4b79-baa5-6cb7323aee5b None None] MFA Rules not processed for user `395c6561137d4ff19da150744f1b1d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:10.118064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c7314433-b284-4b79-baa5-6cb7323aee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:10.196776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1022/2039] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:10.202064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:10.339963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db11811-dc46-4e4d-9d93-0ccf68bbd544 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:10.345120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:10.346013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.346297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.346578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.347312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:10.349349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:10.409140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1018/2040] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:10 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:10.413977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314433-b284-4b79-baa5-6cb7323aee5b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:10.427562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314433-b284-4b79-baa5-6cb7323aee5b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:10.433170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:10.528548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:10.529629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.530028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.530314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.531031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:10.533080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:10.566950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1023/2041] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:10 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:10.571444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:10.583718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae09e42-37d1-48fb-9388-59951193fc58 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:10.589826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:10.590590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.590937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.591289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.592267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:10.594043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:10.607236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1019/2042] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:10.612213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:10.628036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bca695-91f7-4032-b514-b67b11d4922e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:10.636540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:10.637586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.637960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.638293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] PATH_INFO: `/v3/projects/c7f18c1a3ddb4fc6b2a04b862f778a11/users/94991c980405461fa65302023d3ce457/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.656039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f18c1a3ddb4fc6b2a04b862f778a11, user_id=94991c980405461fa65302023d3ce457, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:10.658861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:10.674306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1024/2043] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:10 2022] PUT /identity/v3/projects/c7f18c1a3ddb4fc6b2a04b862f778a11/users/94991c980405461fa65302023d3ce457/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:10.679771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:10.691438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cfc8e-0f79-4ef4-800b-a128c3867117 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:10.748691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:10.777839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aba73-3908-4b86-8e9d-07f55df1e76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:10.784831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aba73-3908-4b86-8e9d-07f55df1e76a admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.784831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aba73-3908-4b86-8e9d-07f55df1e76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.784831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aba73-3908-4b86-8e9d-07f55df1e76a admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.784831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6aba73-3908-4b86-8e9d-07f55df1e76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:10.784831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6aba73-3908-4b86-8e9d-07f55df1e76a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:10.796284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1020/2044] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:04:10.803677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:10.815152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab5de6-cfaa-4829-b358-893ccd078289 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:10.821852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:10.822936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.823337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.823715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] PATH_INFO: `/v3/projects/c7f18c1a3ddb4fc6b2a04b862f778a11/users/94991c980405461fa65302023d3ce457/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.826226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f18c1a3ddb4fc6b2a04b862f778a11, user_id=94991c980405461fa65302023d3ce457, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:10.829014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:10.842600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1025/2045] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:10 2022] PUT /identity/v3/projects/c7f18c1a3ddb4fc6b2a04b862f778a11/users/94991c980405461fa65302023d3ce457/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:10.855911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:10.856282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:10.856629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:10.894904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] MFA Rules not processed for user `94991c980405461fa65302023d3ce457`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:10.896159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:11.007038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1021/2046] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:11.021220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:11.023764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:11.105417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa80f4a-4e68-4774-a343-2f5bc05cdb03 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:11.111238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:11.112188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:11.112671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:11.113193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:11.184066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:11.185577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:11.196808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1026/2047] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:13.039804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:13.041340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:13.054896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb216cc-eedd-4cfb-b222-cf7041dd708e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:13.058604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:13.059275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:13.059575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:13.059848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:13.153745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:13.156403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:13.177433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1022/2048] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:13.985113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:13.997091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc3188-44ed-48dc-9abf-43b49a2c0766 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:14.002466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c3a2a-5ad3-414a-b794-38b495e92294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:14.003281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c3a2a-5ad3-414a-b794-38b495e92294 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:14.003635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c3a2a-5ad3-414a-b794-38b495e92294 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:14.004050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c3a2a-5ad3-414a-b794-38b495e92294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:14.018607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c3a2a-5ad3-414a-b794-38b495e92294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:14.020246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c3a2a-5ad3-414a-b794-38b495e92294 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:14.029973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:14.045269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1027/2049] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:21.037664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:21.172830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a282b06-18f4-4498-8aca-c08bae42d80a service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:21.181626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f635984-e707-4853-9154-78e286525537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:21.183072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f635984-e707-4853-9154-78e286525537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:21.183509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f635984-e707-4853-9154-78e286525537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:21.183812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f635984-e707-4853-9154-78e286525537 admin admin] PATH_INFO: `/v3/users/0022cb978523450ebeba5ceaadd2132b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:21.186013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f635984-e707-4853-9154-78e286525537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0022cb978523450ebeba5ceaadd2132b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:21.189576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f635984-e707-4853-9154-78e286525537 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:21.195094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:21.274631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0022cb978523450ebeba5ceaadd2132b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:21.275676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0022cb978523450ebeba5ceaadd2132b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:21.291570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0022cb978523450ebeba5ceaadd2132b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:21.293905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0022cb978523450ebeba5ceaadd2132b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:21.319408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0022cb978523450ebeba5ceaadd2132b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:21.350870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1023/2050] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:21 2022] DELETE /identity/v3/users/0022cb978523450ebeba5ceaadd2132b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:21.356881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:21.357658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:21.358131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:21.391038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:21.393414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:21.542134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1028/2051] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:21.554651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:21.556528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:21.639513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f635984-e707-4853-9154-78e286525537 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:21.643842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:21.644522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:21.644805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:21.645115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:21.655398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:21.656851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:21.669077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1024/2052] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:22.044283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:22.046630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:22.065290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd543b37-39d9-432f-bba7-eaad8cde1f2a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:22.072109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:22.073332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:22.073746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:22.074222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] PATH_INFO: `/v3/projects/bdd9e1aba3444c0bb815cb0aa45bc7e2` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:22.075904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdd9e1aba3444c0bb815cb0aa45bc7e2)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:22.078500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:22.169181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdd9e1aba3444c0bb815cb0aa45bc7e2', 'request_id': 'req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.184438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdd9e1aba3444c0bb815cb0aa45bc7e2', 'request_id': 'req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.197249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdd9e1aba3444c0bb815cb0aa45bc7e2', 'request_id': 'req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.220756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] The token cache is being invalidate because project bdd9e1aba3444c0bb815cb0aa45bc7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:22.221302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.223070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.226760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1029/2053] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:22 2022] DELETE /identity/v3/projects/bdd9e1aba3444c0bb815cb0aa45bc7e2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:22.232964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:22.249325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cabdc-de0b-4eb6-8464-05858016f064 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:22.342239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:22.343370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:22.343723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:22.344127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] PATH_INFO: `/v3/users/395c6561137d4ff19da150744f1b1d9d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:22.345592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=395c6561137d4ff19da150744f1b1d9d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:22.347692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:22.388846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:22.418796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395c6561137d4ff19da150744f1b1d9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.419646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395c6561137d4ff19da150744f1b1d9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.438056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395c6561137d4ff19da150744f1b1d9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.439042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395c6561137d4ff19da150744f1b1d9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.462467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395c6561137d4ff19da150744f1b1d9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:22.491654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1025/2054] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:22 2022] DELETE /identity/v3/users/395c6561137d4ff19da150744f1b1d9d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:22.931265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:22.933790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:22.970553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a594fb-e76d-4d7e-9c00-ad8b0677cd22 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:23.080099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:23.081016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:23.081318 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:23.081592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] PATH_INFO: `/v3/projects/271400214841469f853aa49b53528f03` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:23.083138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=271400214841469f853aa49b53528f03)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:23.086154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:23.129106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:23.160309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271400214841469f853aa49b53528f03', 'request_id': 'req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.169378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271400214841469f853aa49b53528f03', 'request_id': 'req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.180094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271400214841469f853aa49b53528f03', 'request_id': 'req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.195722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] The token cache is being invalidate because project 271400214841469f853aa49b53528f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:23.196026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.196918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.198808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1030/2055] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:22 2022] DELETE /identity/v3/projects/271400214841469f853aa49b53528f03 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:23.203622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:23.205986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:23.221966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0a7eb-8618-4118-9001-bcac6cf733b1 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:23.320490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:23.322010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:23.322010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:23.323492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] PATH_INFO: `/v3/users/94991c980405461fa65302023d3ce457` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:23.325435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=94991c980405461fa65302023d3ce457)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:23.327808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:23.422570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94991c980405461fa65302023d3ce457'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.423478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94991c980405461fa65302023d3ce457'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.438686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94991c980405461fa65302023d3ce457'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.439836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94991c980405461fa65302023d3ce457'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.479043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f735f182-4f44-4ff4-b0f4-44b7817860dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94991c980405461fa65302023d3ce457'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:23.539170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1026/2056] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:23 2022] DELETE /identity/v3/users/94991c980405461fa65302023d3ce457 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:24.028086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:24.030227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:24.060362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c763b1-f94f-4072-8b97-3aeb1e5a36eb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:24.151717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:24.153128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:24.153563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:24.153979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] PATH_INFO: `/v3/projects/c7f18c1a3ddb4fc6b2a04b862f778a11` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:24.156031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7f18c1a3ddb4fc6b2a04b862f778a11)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:24.159213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:24.236136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7f18c1a3ddb4fc6b2a04b862f778a11', 'request_id': 'req-72467c35-f111-48ad-99c1-3ae589105b0a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:24.247246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7f18c1a3ddb4fc6b2a04b862f778a11', 'request_id': 'req-72467c35-f111-48ad-99c1-3ae589105b0a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:24.259478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7f18c1a3ddb4fc6b2a04b862f778a11', 'request_id': 'req-72467c35-f111-48ad-99c1-3ae589105b0a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:24.276437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] The token cache is being invalidate because project c7f18c1a3ddb4fc6b2a04b862f778a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:24.276754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:24.277658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-72467c35-f111-48ad-99c1-3ae589105b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:24.279861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1031/2057] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:24 2022] DELETE /identity/v3/projects/c7f18c1a3ddb4fc6b2a04b862f778a11 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:24.288525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766869b-1190-493d-acc6-1318c404a44f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:24.289035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766869b-1190-493d-acc6-1318c404a44f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:24.289536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766869b-1190-493d-acc6-1318c404a44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:24.320496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e766869b-1190-493d-acc6-1318c404a44f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:24.324221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e766869b-1190-493d-acc6-1318c404a44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:24.355206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:24.448229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1027/2058] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:24.454072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0eebd-3289-433c-ad7c-57cd90d06c43 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:24.454372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0eebd-3289-433c-ad7c-57cd90d06c43 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:24.454697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0eebd-3289-433c-ad7c-57cd90d06c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:24.480059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-64b0eebd-3289-433c-ad7c-57cd90d06c43 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:24.498735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-64b0eebd-3289-433c-ad7c-57cd90d06c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:24.515702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1032/2059] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:24.516323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766869b-1190-493d-acc6-1318c404a44f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:24.546565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766869b-1190-493d-acc6-1318c404a44f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:24.555376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:24.556791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:24.557395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:24.557939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:24.558904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:24.559465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:24.563124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:24.584521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1028/2060] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:24.586292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0eebd-3289-433c-ad7c-57cd90d06c43 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:24.606234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0eebd-3289-433c-ad7c-57cd90d06c43 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:24.617784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:24.619363 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:24.619875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:24.620750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:24.623066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:24.626998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:24.679396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1033/2061] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:24 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:24.684347 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:24.703379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be3b-6902-4092-a32d-c91632165bfa admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:24.893319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:24.894384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:24.895002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:24.895451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:24.896608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:24.899549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:24.934165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1029/2062] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:24 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:24.939031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:24.962817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89afa23-d1b3-4a11-b626-a2d630b0bb6d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:24.972297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:24.974239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:24.975242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:24.975827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:24.977999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:24.994794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:25.018507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1034/2063] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:25.026590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:25.054138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13fdbc-83c3-40d3-9f90-a9b3099557dc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:25.065061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbdb6a-578f-435b-90f4-7d97cca2c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:25.066717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbdb6a-578f-435b-90f4-7d97cca2c6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:25.067643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbdb6a-578f-435b-90f4-7d97cca2c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:25.069455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbdb6a-578f-435b-90f4-7d97cca2c6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:25.070956 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbdb6a-578f-435b-90f4-7d97cca2c6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:25.072790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbdb6a-578f-435b-90f4-7d97cca2c6b8 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:25.087779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1030/2064] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:25.094314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:25.104414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:25.114294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f9364-ff6f-40e4-ad5d-74cfbf171f3c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:25.125164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:25.127121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:25.127491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:25.127843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] PATH_INFO: `/v3/projects/d40d91112df648b7bc629f59a59705ac/users/686552536d9d45f387ef4e972d5bacdd/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:25.149142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40d91112df648b7bc629f59a59705ac, user_id=686552536d9d45f387ef4e972d5bacdd, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:25.152144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:25.182438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1035/2065] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:25 2022] PUT /identity/v3/projects/d40d91112df648b7bc629f59a59705ac/users/686552536d9d45f387ef4e972d5bacdd/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:25.202792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:25.203085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:25.203370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:25.254748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] MFA Rules not processed for user `686552536d9d45f387ef4e972d5bacdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:25.256702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:25.372431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1031/2066] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:25.385269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:25.388646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:25.499775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a157319-07fa-4cee-97d9-107ea86fcdbd admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:25.506073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471fc77-107f-4883-b506-354e1241a40e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:25.506073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471fc77-107f-4883-b506-354e1241a40e service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:25.506073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471fc77-107f-4883-b506-354e1241a40e service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:25.506073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471fc77-107f-4883-b506-354e1241a40e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:25.517419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471fc77-107f-4883-b506-354e1241a40e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:25.519071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471fc77-107f-4883-b506-354e1241a40e service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:25.531454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1036/2067] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:26.142005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:26.144261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:26.182748 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:26.291292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ec140-c469-4c08-bcf8-869663c31261 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:26.301432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:26.303716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:26.304071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:26.304578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] PATH_INFO: `/v3/users/52837a163874423785ca6fcda7765d57` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:26.306301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=52837a163874423785ca6fcda7765d57)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:26.308550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:26.382876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52837a163874423785ca6fcda7765d57'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:26.383904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52837a163874423785ca6fcda7765d57'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:26.394611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52837a163874423785ca6fcda7765d57'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:26.396398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52837a163874423785ca6fcda7765d57'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:26.413757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52837a163874423785ca6fcda7765d57'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:26.437013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1032/2068] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:26 2022] DELETE /identity/v3/users/52837a163874423785ca6fcda7765d57 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:26.441886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:26.444201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:26.446203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:26.470782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:26.472856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:26.605125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1037/2069] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:26.616754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:26.622149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:26.778846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca6e8c-eb30-4575-8a79-6d4be6072eea admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:26.784259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ed927-73b6-4acc-9188-744f38b6aeb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:26.785045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ed927-73b6-4acc-9188-744f38b6aeb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:26.785400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ed927-73b6-4acc-9188-744f38b6aeb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:26.785954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ed927-73b6-4acc-9188-744f38b6aeb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:26.799342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ed927-73b6-4acc-9188-744f38b6aeb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:26.801433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ed927-73b6-4acc-9188-744f38b6aeb4 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:26.818561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1033/2070] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:27.276004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:27.285413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:27.308342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de47a8-5f13-4f05-a81d-01a638eb9479 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:27.314380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:27.316220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:27.317046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:27.318919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] PATH_INFO: `/v3/users/30b4b62251104fc98d638c0b8561b602` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:27.320501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30b4b62251104fc98d638c0b8561b602)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:27.322756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:27.454096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30b4b62251104fc98d638c0b8561b602'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:27.486599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30b4b62251104fc98d638c0b8561b602'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:27.488253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30b4b62251104fc98d638c0b8561b602'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:27.510222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30b4b62251104fc98d638c0b8561b602'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:27.512567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30b4b62251104fc98d638c0b8561b602'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:27.533253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:27.574834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1038/2071] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:27 2022] DELETE /identity/v3/users/30b4b62251104fc98d638c0b8561b602 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:27.583368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:27.584934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:27.585445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:27.625183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:27.627026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:27.650672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:27.658121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:27.749381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:27.798970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1034/2072] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:27.807411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89368a29-1acd-4c6c-934a-c8fd722b1c28 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:27.815624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:27.817324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:27.819327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:27.819622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:27.819930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] PATH_INFO: `/v3/projects/ef4bdc1e6e07411fac23ea832d862204` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:27.821558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4bdc1e6e07411fac23ea832d862204)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:27.822623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:27.823702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:27.990197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4bdc1e6e07411fac23ea832d862204', 'request_id': 'req-6725f24a-04c0-43d8-9f32-1ba91ac966c2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:28.007288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c2cb9-317f-4c7b-a612-20c6d21a957c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:28.009479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4bdc1e6e07411fac23ea832d862204', 'request_id': 'req-6725f24a-04c0-43d8-9f32-1ba91ac966c2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:28.024435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4bdc1e6e07411fac23ea832d862204', 'request_id': 'req-6725f24a-04c0-43d8-9f32-1ba91ac966c2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:28.096215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] The token cache is being invalidate because project ef4bdc1e6e07411fac23ea832d862204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:28.096752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:28.098603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f90cc-6b1f-4b35-adfd-7fdc9d8c89e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:28.101969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f90cc-6b1f-4b35-adfd-7fdc9d8c89e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:28.102442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f90cc-6b1f-4b35-adfd-7fdc9d8c89e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:28.102561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6725f24a-04c0-43d8-9f32-1ba91ac966c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:28.102921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f90cc-6b1f-4b35-adfd-7fdc9d8c89e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:28.112944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-126f90cc-6b1f-4b35-adfd-7fdc9d8c89e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:28.119773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1039/2073] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:27 2022] DELETE /identity/v3/projects/ef4bdc1e6e07411fac23ea832d862204 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:28.122713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789770ff-d3a8-415a-a4e9-366d39102bda None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:28.123245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789770ff-d3a8-415a-a4e9-366d39102bda None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:28.123806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789770ff-d3a8-415a-a4e9-366d39102bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:28.183802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-789770ff-d3a8-415a-a4e9-366d39102bda None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:28.185068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-789770ff-d3a8-415a-a4e9-366d39102bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:28.264028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f90cc-6b1f-4b35-adfd-7fdc9d8c89e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:28.271763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f90cc-6b1f-4b35-adfd-7fdc9d8c89e8 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:28.292914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1035/2074] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:28.314551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1040/2075] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:28.328696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176db4a6-4280-4831-9d52-6e8d8f13d7dc None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:28.328696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176db4a6-4280-4831-9d52-6e8d8f13d7dc None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:28.328696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176db4a6-4280-4831-9d52-6e8d8f13d7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:28.393441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-176db4a6-4280-4831-9d52-6e8d8f13d7dc None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:28.393441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-176db4a6-4280-4831-9d52-6e8d8f13d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:28.430498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789770ff-d3a8-415a-a4e9-366d39102bda None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:28.433534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1036/2076] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:28.444916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789770ff-d3a8-415a-a4e9-366d39102bda None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:28.472104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:28.475131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:28.475700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:28.476159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:28.478762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:28.498849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:28.505735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:28.548502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1041/2077] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:28.554720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176db4a6-4280-4831-9d52-6e8d8f13d7dc None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:28.578049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176db4a6-4280-4831-9d52-6e8d8f13d7dc None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:28.597820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:28.598531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:28.598865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:28.599205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:28.600335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:28.602390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:28.670565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1037/2078] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:28 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:28.672164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:28.694530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb4ac3-88b2-4f52-95dc-b46381a76bb8 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:28.717611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:28.773152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:28.776684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:28.861552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:28.910847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:28.912292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:28.912653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:28.913110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:28.913848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:28.916431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:28.966235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1042/2079] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:28 2022] POST /identity/v3/users => generated 291 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:28.980330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:28.993013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48106c-21cb-4333-903c-31ce1e160d69 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:28.999198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.000121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.000528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.000874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] PATH_INFO: `/v3/users/686552536d9d45f387ef4e972d5bacdd` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.015127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] RBAC: Authorizing `identity:delete_user(user_id=686552536d9d45f387ef4e972d5bacdd)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.018121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.035278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49935-6871-445d-be01-831fb506fc85 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:29.042273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.042583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.042803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.043122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.043889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.045676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.058357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1043/2080] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:29.070702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:29.082857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4aa79-0b9c-4aa5-b5e3-3bb59422f5dc admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:29.090059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.090284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.090560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.090842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.091593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.093304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.097924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686552536d9d45f387ef4e972d5bacdd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.099892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686552536d9d45f387ef4e972d5bacdd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.113933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1044/2081] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:04:29.116809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686552536d9d45f387ef4e972d5bacdd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.118569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686552536d9d45f387ef4e972d5bacdd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.120721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:29.122631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.137280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9af71-8ff9-4229-b51f-ecf4a562fd73 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:29.142711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5daf8-68cc-48eb-882f-3a06847b2ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.143632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5daf8-68cc-48eb-882f-3a06847b2ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.143920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5daf8-68cc-48eb-882f-3a06847b2ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.144200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5daf8-68cc-48eb-882f-3a06847b2ca5 admin admin] PATH_INFO: `/v3/projects/7bdadf97d44048c3bec51c0ae58ee4a6/users/c1feaba97f594ffda9dd2962bebdc082/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.149620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4a860cb6-ccf8-434a-a91b-2e48a4d7750c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686552536d9d45f387ef4e972d5bacdd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.177197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5daf8-68cc-48eb-882f-3a06847b2ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bdadf97d44048c3bec51c0ae58ee4a6, user_id=c1feaba97f594ffda9dd2962bebdc082, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.179350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1038/2082] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:28 2022] DELETE /identity/v3/users/686552536d9d45f387ef4e972d5bacdd => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:29.186099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5daf8-68cc-48eb-882f-3a06847b2ca5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.187023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.187619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.188235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.211872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1045/2083] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:29 2022] PUT /identity/v3/projects/7bdadf97d44048c3bec51c0ae58ee4a6/users/c1feaba97f594ffda9dd2962bebdc082/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:29.219142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.220407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.220886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.225725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:29.227556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.263982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:29.265992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.358174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1039/2084] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:29.387159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1046/2085] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:29.392782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:29.393098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:29.394846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.396167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.411491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648d16-c4c2-42ba-9801-1006bb253da0 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:29.417476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.418516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.418809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.419095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] PATH_INFO: `/v3/users/0800a22846504b0d9c49e1520a69e45b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.420457 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0800a22846504b0d9c49e1520a69e45b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.422623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.478744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:29.489019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0800a22846504b0d9c49e1520a69e45b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.501065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:29.513538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0800a22846504b0d9c49e1520a69e45b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.514592 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0800a22846504b0d9c49e1520a69e45b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.518153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d938f8-4149-4d0e-8438-56427b1bf204 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:29.524164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.524933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.525267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.525541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.526298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0800a22846504b0d9c49e1520a69e45b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.527209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0800a22846504b0d9c49e1520a69e45b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.527804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.546424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.547394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1047/2086] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:29 2022] DELETE /identity/v3/users/0800a22846504b0d9c49e1520a69e45b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:29.552483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.552803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:29.553671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.569330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1040/2087] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:29.570837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:29.573023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.600886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f64b-b2cc-43a3-a4a0-2f16b0f58e36 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:29.633148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddda65-c28b-4441-887f-4844b3b3eeb1 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:29.643035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.643035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.643035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.643035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.647905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:29.703427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:29.704332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:29.704615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:29.705117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] PATH_INFO: `/v3/projects/473bb304cb3b4dedab80f534d490e2ef` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:29.707263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=473bb304cb3b4dedab80f534d490e2ef)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.711364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.820297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473bb304cb3b4dedab80f534d490e2ef', 'request_id': 'req-289c1962-517c-487e-9461-459cfe240b60'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.827245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:29.830257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:29.840449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473bb304cb3b4dedab80f534d490e2ef', 'request_id': 'req-289c1962-517c-487e-9461-459cfe240b60'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.857258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1048/2088] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:29.863352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473bb304cb3b4dedab80f534d490e2ef', 'request_id': 'req-289c1962-517c-487e-9461-459cfe240b60'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.893149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] The token cache is being invalidate because project 473bb304cb3b4dedab80f534d490e2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:29.894169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.896547 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:29.901565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1041/2089] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:29 2022] DELETE /identity/v3/projects/473bb304cb3b4dedab80f534d490e2ef => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:29.906654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:29.923475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30878108-969b-4434-8add-44b62513d6ca service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:30.040581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:30.042371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:30.043216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:30.043891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] PATH_INFO: `/v3/users/91f2c92618fd429db76f2288e8b63edd` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:30.046708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91f2c92618fd429db76f2288e8b63edd)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:30.052240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:30.071294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:30.073154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:30.095703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:30.109774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c1962-517c-487e-9461-459cfe240b60 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:30.126661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:30.127676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:30.128126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:30.128420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] PATH_INFO: `/v3/projects/f46c6cdc3217456dba6eda8ff23e9f3a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:30.131326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46c6cdc3217456dba6eda8ff23e9f3a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:30.134649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:30.169282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91f2c92618fd429db76f2288e8b63edd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.186397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91f2c92618fd429db76f2288e8b63edd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.187262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91f2c92618fd429db76f2288e8b63edd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.200619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91f2c92618fd429db76f2288e8b63edd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.202075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1aeadc0f-2d7c-446f-bca7-59a6ebf4094e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91f2c92618fd429db76f2288e8b63edd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.228034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1049/2090] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:29 2022] DELETE /identity/v3/users/91f2c92618fd429db76f2288e8b63edd => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:30.228931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46c6cdc3217456dba6eda8ff23e9f3a', 'request_id': 'req-e8114c9c-b2ef-4da7-8619-c272d5245f64'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.247194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46c6cdc3217456dba6eda8ff23e9f3a', 'request_id': 'req-e8114c9c-b2ef-4da7-8619-c272d5245f64'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.269494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46c6cdc3217456dba6eda8ff23e9f3a', 'request_id': 'req-e8114c9c-b2ef-4da7-8619-c272d5245f64'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.272167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:30.325774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] The token cache is being invalidate because project f46c6cdc3217456dba6eda8ff23e9f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:30.326330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.327275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.329355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1042/2091] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:30 2022] DELETE /identity/v3/projects/f46c6cdc3217456dba6eda8ff23e9f3a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:30.337834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8251c7-b697-4918-8457-98018ee69a0a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:30.338332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8251c7-b697-4918-8457-98018ee69a0a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:30.338898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8251c7-b697-4918-8457-98018ee69a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:30.418334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bc8251c7-b697-4918-8457-98018ee69a0a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:30.424239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bc8251c7-b697-4918-8457-98018ee69a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:30.527175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:30.529631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:30.662971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1050/2092] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:30.676728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1bcf6-b89d-48a8-a24a-577765ba54c3 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:30.677471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1bcf6-b89d-48a8-a24a-577765ba54c3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:30.677750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1bcf6-b89d-48a8-a24a-577765ba54c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:30.717289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-65a1bcf6-b89d-48a8-a24a-577765ba54c3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:30.718942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-65a1bcf6-b89d-48a8-a24a-577765ba54c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:30.721435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8114c9c-b2ef-4da7-8619-c272d5245f64 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:30.732671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:30.733702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:30.733989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:30.734861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] PATH_INFO: `/v3/projects/d40d91112df648b7bc629f59a59705ac` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:30.740445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40d91112df648b7bc629f59a59705ac)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:30.741375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1051/2093] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:30.743720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:30.748092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1bcf6-b89d-48a8-a24a-577765ba54c3 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:30.765955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1bcf6-b89d-48a8-a24a-577765ba54c3 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:30.777233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:30.778238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:30.778695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:30.779134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:30.782707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:30.783229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:30.785081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:30.814697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1052/2094] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:30.823566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:30.850329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad153a8-c6bf-46c1-a13d-dae1e57243b9 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:30.867304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40d91112df648b7bc629f59a59705ac', 'request_id': 'req-351cc158-49dd-4784-badd-61415a0fc68b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.881928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40d91112df648b7bc629f59a59705ac', 'request_id': 'req-351cc158-49dd-4784-badd-61415a0fc68b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.897481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40d91112df648b7bc629f59a59705ac', 'request_id': 'req-351cc158-49dd-4784-badd-61415a0fc68b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.938784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] The token cache is being invalidate because project d40d91112df648b7bc629f59a59705ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:30.939057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.940630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-351cc158-49dd-4784-badd-61415a0fc68b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:30.950174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1043/2095] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:30 2022] DELETE /identity/v3/projects/d40d91112df648b7bc629f59a59705ac => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:30.951817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:30.952098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:30.952378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:30.996803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:31.001145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.012779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ac05b-f058-43d1-a0a3-1fa7f14b2cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.013979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ac05b-f058-43d1-a0a3-1fa7f14b2cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.014650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ac05b-f058-43d1-a0a3-1fa7f14b2cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.014650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ac05b-f058-43d1-a0a3-1fa7f14b2cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.016336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ac05b-f058-43d1-a0a3-1fa7f14b2cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.018964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ac05b-f058-43d1-a0a3-1fa7f14b2cf1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:31.059241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1044/2096] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:31.061989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.069616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.077385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1053/2097] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:30 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:31.079846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.079846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.079846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.088898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:31.100482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db4da-8408-44c4-af35-84c166a86c9a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.112814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:31.115500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.221375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.225789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.226447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.226741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] PATH_INFO: `/v3/projects/517e03f54eff47039a3e52ae030c94bc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.228235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517e03f54eff47039a3e52ae030c94bc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.231742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:31.251112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1054/2098] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:31.251981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.257936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.280889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:31.305978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7d160-489d-4297-8cfc-2b0d858718ee None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.338721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517e03f54eff47039a3e52ae030c94bc', 'request_id': 'req-02c23898-3192-472d-8ada-8e89ced7a8f4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:31.346207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517e03f54eff47039a3e52ae030c94bc', 'request_id': 'req-02c23898-3192-472d-8ada-8e89ced7a8f4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:31.358055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517e03f54eff47039a3e52ae030c94bc', 'request_id': 'req-02c23898-3192-472d-8ada-8e89ced7a8f4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:31.388805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] The token cache is being invalidate because project 517e03f54eff47039a3e52ae030c94bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:31.389267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:31.390503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:31.395227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1045/2099] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:31 2022] DELETE /identity/v3/projects/517e03f54eff47039a3e52ae030c94bc => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:31.397775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.413019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.491177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8a366-ce40-479c-a9f9-64d5e0efddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.492462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8a366-ce40-479c-a9f9-64d5e0efddf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.492585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8a366-ce40-479c-a9f9-64d5e0efddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.494478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8a366-ce40-479c-a9f9-64d5e0efddf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.498399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8a366-ce40-479c-a9f9-64d5e0efddf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.506377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8a366-ce40-479c-a9f9-64d5e0efddf5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:31.535677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1055/2100] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:31 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:31.537655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ed433-84e1-4205-ab9d-473c230681fe None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.537962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ed433-84e1-4205-ab9d-473c230681fe None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.538221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ed433-84e1-4205-ab9d-473c230681fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.561712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-224ed433-84e1-4205-ab9d-473c230681fe None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:31.563194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-224ed433-84e1-4205-ab9d-473c230681fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.574957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1056/2101] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:31.579132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccc2d6-9300-44eb-910c-ee408396b2b6 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.579724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccc2d6-9300-44eb-910c-ee408396b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.580186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccc2d6-9300-44eb-910c-ee408396b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.581597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c23898-3192-472d-8ada-8e89ced7a8f4 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.592248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.593909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.594511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.595235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.596991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:31.598219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.604391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:31.623721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-abccc2d6-9300-44eb-910c-ee408396b2b6 None None] MFA Rules not processed for user `c1feaba97f594ffda9dd2962bebdc082`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:31.625879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-abccc2d6-9300-44eb-910c-ee408396b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.636551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1046/2102] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:31.638311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.644535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.670679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59426d30-8bd5-44c0-9b51-e785d11803b6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.680758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391f254-6781-4462-a2eb-8b009ca4ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.682356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391f254-6781-4462-a2eb-8b009ca4ff17 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.682991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391f254-6781-4462-a2eb-8b009ca4ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.683551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391f254-6781-4462-a2eb-8b009ca4ff17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.685239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391f254-6781-4462-a2eb-8b009ca4ff17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.688932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391f254-6781-4462-a2eb-8b009ca4ff17 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:31.734115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1047/2103] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:31.738155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14affb-82f4-4bf1-8511-9e913b52a181 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.738687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14affb-82f4-4bf1-8511-9e913b52a181 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.739178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14affb-82f4-4bf1-8511-9e913b52a181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.741496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1057/2104] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:31.743012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccc2d6-9300-44eb-910c-ee408396b2b6 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.781378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cd14affb-82f4-4bf1-8511-9e913b52a181 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:31.783619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccc2d6-9300-44eb-910c-ee408396b2b6 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.783788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cd14affb-82f4-4bf1-8511-9e913b52a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.793531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.794711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.795224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.795732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.797128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.799869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1048/2105] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:31.801239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14affb-82f4-4bf1-8511-9e913b52a181 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.801838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:31.824126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14affb-82f4-4bf1-8511-9e913b52a181 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.828962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:31.835035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.836458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.837383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.837806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.839248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.842477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:31.846722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1058/2106] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:31 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:31.848155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.851788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:31.868627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1049/2107] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:31.870021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:31.887163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f45aa-3a67-465e-8406-ef6c1f8f6a14 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.926701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e0d4b-f6c5-47f0-9148-73ce3f93c356 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:31.931838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.932620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.932915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.933241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.993257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:31.993962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:31.994257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:31.994537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:31.995294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:31.995610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:31.997298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.010745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1050/2108] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:32.010745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.014618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.014618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.022565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611ce62-7c37-4d0a-bac1-f3d0dc88e818 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.025450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1059/2109] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:32.026384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.027997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.028744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.029013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.029317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.030099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.032113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.040355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:32.043402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980921fa-50bf-4055-a9b5-65df20f0e24d service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.053956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.055394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.055802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.056092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] PATH_INFO: `/v3/projects/69e2bbfe423846dc9755f233e89ecc8f/users/719204a59b4c4344adb0d1b4c698003e/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.073583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1051/2110] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:32 2022] POST /identity/v3/users => generated 292 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:32.074819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.088529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e2bbfe423846dc9755f233e89ecc8f, user_id=719204a59b4c4344adb0d1b4c698003e, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.091335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.094076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb86742-59ff-46ba-a561-8718881520a0 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.101734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd49d08-bfaf-4693-b9c9-cf9c1554be3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.102805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd49d08-bfaf-4693-b9c9-cf9c1554be3a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.103350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd49d08-bfaf-4693-b9c9-cf9c1554be3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.103790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd49d08-bfaf-4693-b9c9-cf9c1554be3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.104915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd49d08-bfaf-4693-b9c9-cf9c1554be3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.107776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd49d08-bfaf-4693-b9c9-cf9c1554be3a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.108194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1060/2111] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:32 2022] PUT /identity/v3/projects/69e2bbfe423846dc9755f233e89ecc8f/users/719204a59b4c4344adb0d1b4c698003e/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:32.109213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.164787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1052/2112] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:32 2022] POST /identity/v3/projects => generated 363 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:32.164787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dcd51-2bee-4127-96cd-0714f5ae0fd5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.164787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dcd51-2bee-4127-96cd-0714f5ae0fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.164787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dcd51-2bee-4127-96cd-0714f5ae0fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.199890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1144c5-22a4-4820-b11a-a926e49a0969 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.201168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f71dcd51-2bee-4127-96cd-0714f5ae0fd5 None None] MFA Rules not processed for user `719204a59b4c4344adb0d1b4c698003e`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:32.203338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f71dcd51-2bee-4127-96cd-0714f5ae0fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:32.242990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:32.277144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.277931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.278325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.278670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.279681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.281499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.293867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1061/2113] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:32.294822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.307748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079aad2f-db8f-4955-87a7-ba0b29e00e71 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.314588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.315333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.315625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.316421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.316782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.319180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.346624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1053/2114] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:32.347575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dcd51-2bee-4127-96cd-0714f5ae0fd5 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.364114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1062/2115] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:32 2022] POST /identity/v3/users => generated 282 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:32.369079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dcd51-2bee-4127-96cd-0714f5ae0fd5 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.372306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.376712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:32.378240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.379535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.381530 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.381883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.382842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.385132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.402254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1054/2116] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:32.403496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.435305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f1934-bb9b-474f-8c82-bd2a7fc17b9e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.447073 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.448040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.449173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.449604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.450947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.453978 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.473141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1055/2117] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:32.473141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.484402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e493f-9f59-4954-bb25-72174e821da2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.490832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b76484-ee81-429f-8b6f-31d99a9bfa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.491763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76484-ee81-429f-8b6f-31d99a9bfa6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.492052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76484-ee81-429f-8b6f-31d99a9bfa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.492335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76484-ee81-429f-8b6f-31d99a9bfa6c admin admin] PATH_INFO: `/v3/projects/a7afb1bdb06042f4b983631745f32e19/users/6663857e60ca4440a9a620007be4c2c7/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.507050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7023d-2aa7-4504-a7f6-f83d52ee15fe admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.513975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.514382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.514729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.515004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.525551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.527211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.543778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b76484-ee81-429f-8b6f-31d99a9bfa6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7afb1bdb06042f4b983631745f32e19, user_id=6663857e60ca4440a9a620007be4c2c7, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.557358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1063/2118] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:32.557358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.557556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b76484-ee81-429f-8b6f-31d99a9bfa6c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.569224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc6141-5b55-40f9-b137-e0cc94e36a6c service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.575123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.575912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.576422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.576776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.577636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.579676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.582041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1056/2119] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:32 2022] PUT /identity/v3/projects/a7afb1bdb06042f4b983631745f32e19/users/6663857e60ca4440a9a620007be4c2c7/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:32.588228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.588706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.589118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.591611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1064/2120] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:32.596486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.610577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22c3c5-3ae7-4b7b-9312-ecec61ef3922 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.613116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:32.615149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:32.700061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1057/2121] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:32.731021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:32.732519 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:32.735515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05df07-4f2c-4fb1-a9c0-c50533fbe952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.737065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05df07-4f2c-4fb1-a9c0-c50533fbe952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.737581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05df07-4f2c-4fb1-a9c0-c50533fbe952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.738104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05df07-4f2c-4fb1-a9c0-c50533fbe952 admin admin] PATH_INFO: `/v3/projects/d578fa093a8a4a1bba9c3168e5cf8fe5/users/4c0c433bddab4e54a4bdf3494689f8c4/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.750114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:32.772154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05df07-4f2c-4fb1-a9c0-c50533fbe952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d578fa093a8a4a1bba9c3168e5cf8fe5, user_id=4c0c433bddab4e54a4bdf3494689f8c4, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:32.778085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05df07-4f2c-4fb1-a9c0-c50533fbe952 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:32.807832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1065/2122] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:32 2022] PUT /identity/v3/projects/d578fa093a8a4a1bba9c3168e5cf8fe5/users/4c0c433bddab4e54a4bdf3494689f8c4/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:32.823345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013f8ee-5072-49c1-9e00-ce6d111b5de2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.823653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013f8ee-5072-49c1-9e00-ce6d111b5de2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.824111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013f8ee-5072-49c1-9e00-ce6d111b5de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.850470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fcf19-7641-4b93-931c-216dda811fbe None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:32.856608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:32.870917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d013f8ee-5072-49c1-9e00-ce6d111b5de2 None None] MFA Rules not processed for user `4c0c433bddab4e54a4bdf3494689f8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:32.872234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d013f8ee-5072-49c1-9e00-ce6d111b5de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:32.950349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:32.951345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:32.951820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:32.952290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:32.979717 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1066/2123] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:32.987085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013f8ee-5072-49c1-9e00-ce6d111b5de2 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:33.095765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013f8ee-5072-49c1-9e00-ce6d111b5de2 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:33.100865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:33.102478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:33.103428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.104019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.104019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.104910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:33.107179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:33.107416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:33.135968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1058/2124] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:33.165014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1067/2125] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:32 2022] POST /identity/v3/projects => generated 369 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:33.167571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:33.184395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d2b8-94f5-4a12-af1a-d0b3b907fe68 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:33.288705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:33.290346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.292087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.292693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.294176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:33.297416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:33.344502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1059/2126] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:33 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:33.347913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:33.365447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54adc-da3c-4488-b2b4-481a7fa4de66 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:33.381384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:33.382098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.382381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.382655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.383423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:33.386926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:33.410817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1068/2127] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:33.419124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:33.472213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11969bc8-0a8b-4c01-bfba-c513db6f4b66 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:33.500733 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec51c-5436-4212-ad02-1fba64684e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:33.501809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec51c-5436-4212-ad02-1fba64684e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.502198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec51c-5436-4212-ad02-1fba64684e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.502608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec51c-5436-4212-ad02-1fba64684e31 admin admin] PATH_INFO: `/v3/projects/52d7a90da0924639b7c7fcd06dcbd058/users/5313f89806b94c2080a3ec1f461c1f3a/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.506598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:33.525534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec51c-5436-4212-ad02-1fba64684e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d7a90da0924639b7c7fcd06dcbd058, user_id=5313f89806b94c2080a3ec1f461c1f3a, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:33.530030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec51c-5436-4212-ad02-1fba64684e31 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:33.551163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1060/2128] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:33 2022] PUT /identity/v3/projects/52d7a90da0924639b7c7fcd06dcbd058/users/5313f89806b94c2080a3ec1f461c1f3a/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:33.555741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:33.577766 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff452d1-17e0-4f68-9fef-6e36ebd82023 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:33.643408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.643944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.644365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.683953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] MFA Rules not processed for user `c1feaba97f594ffda9dd2962bebdc082`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:33.686012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:33.686220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:33.686782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.687065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.687353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.688116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:33.690449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:33.711660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1069/2129] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:04:33.713201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:33.727364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e858e-dc06-4959-9dbb-e99b8e253ccd admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:33.733977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f3574-b24b-48f7-92d4-3475c1e8e48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:33.734987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f3574-b24b-48f7-92d4-3475c1e8e48b admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.735347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f3574-b24b-48f7-92d4-3475c1e8e48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.735700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f3574-b24b-48f7-92d4-3475c1e8e48b admin admin] PATH_INFO: `/v3/projects/52d7a90da0924639b7c7fcd06dcbd058/users/5313f89806b94c2080a3ec1f461c1f3a/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.738218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f3574-b24b-48f7-92d4-3475c1e8e48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d7a90da0924639b7c7fcd06dcbd058, user_id=5313f89806b94c2080a3ec1f461c1f3a, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:33.742137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f3574-b24b-48f7-92d4-3475c1e8e48b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:33.746941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:33.761059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1070/2130] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:33 2022] PUT /identity/v3/projects/52d7a90da0924639b7c7fcd06dcbd058/users/5313f89806b94c2080a3ec1f461c1f3a/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:33.773718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590f84d-74fe-4d95-850e-7379cc5c1f90 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:33.774361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590f84d-74fe-4d95-850e-7379cc5c1f90 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:33.774575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590f84d-74fe-4d95-850e-7379cc5c1f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:33.827992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8590f84d-74fe-4d95-850e-7379cc5c1f90 None None] MFA Rules not processed for user `5313f89806b94c2080a3ec1f461c1f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:33.829536 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8590f84d-74fe-4d95-850e-7379cc5c1f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:33.861779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1061/2131] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:33.973769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1071/2132] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:33 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:33.990876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:33.993190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:34.142912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a30cc5-d80c-4823-a0e0-be6d4505436a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:34.251846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23f6b9-20b7-462f-a746-ba98ecf45a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:34.253210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23f6b9-20b7-462f-a746-ba98ecf45a69 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:34.253866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23f6b9-20b7-462f-a746-ba98ecf45a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:34.254173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23f6b9-20b7-462f-a746-ba98ecf45a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:34.267562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23f6b9-20b7-462f-a746-ba98ecf45a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:34.269305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23f6b9-20b7-462f-a746-ba98ecf45a69 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:34.283234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1062/2133] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:33 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:37.433061 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590f84d-74fe-4d95-850e-7379cc5c1f90 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:37.538109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590f84d-74fe-4d95-850e-7379cc5c1f90 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:37.543291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:37.545783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:37.546076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:37.546352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] PATH_INFO: `/v3/users/4c0c433bddab4e54a4bdf3494689f8c4` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:37.547847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0c433bddab4e54a4bdf3494689f8c4)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:37.549851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:37.638244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0c433bddab4e54a4bdf3494689f8c4'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:37.670984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0c433bddab4e54a4bdf3494689f8c4'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:37.671757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0c433bddab4e54a4bdf3494689f8c4'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:37.682972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0c433bddab4e54a4bdf3494689f8c4'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:37.684358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0c433bddab4e54a4bdf3494689f8c4'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:37.701139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:37.709086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1072/2134] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:37 2022] DELETE /identity/v3/users/4c0c433bddab4e54a4bdf3494689f8c4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:37.718373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:37.719296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:37.719585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:37.741531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:37.742873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:37.837267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:37.873996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1063/2135] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:37.889034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:37.891059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:37.968560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be2c4e-d693-4273-8420-d38ada097a74 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:37.973058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:37.974385 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:37.974785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:37.975125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:37.986045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:37.987644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:37.999836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1073/2136] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:38.499176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:38.503138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:38.559704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5caeb9-33fb-4f2b-aaf4-1304b6659009 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:38.568866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:38.569334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:38.569618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:38.570078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] PATH_INFO: `/v3/projects/d578fa093a8a4a1bba9c3168e5cf8fe5` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:38.572555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d578fa093a8a4a1bba9c3168e5cf8fe5)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:38.575037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:38.660635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd578fa093a8a4a1bba9c3168e5cf8fe5', 'request_id': 'req-d978dd63-678c-452e-ba57-20d1568d587e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:38.675953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd578fa093a8a4a1bba9c3168e5cf8fe5', 'request_id': 'req-d978dd63-678c-452e-ba57-20d1568d587e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:38.704752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd578fa093a8a4a1bba9c3168e5cf8fe5', 'request_id': 'req-d978dd63-678c-452e-ba57-20d1568d587e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:38.736749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] The token cache is being invalidate because project d578fa093a8a4a1bba9c3168e5cf8fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:38.739642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:38.740676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:38.745874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1064/2137] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:38 2022] DELETE /identity/v3/projects/d578fa093a8a4a1bba9c3168e5cf8fe5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:38.752232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:38.765333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6feeb-ca0d-42e3-a2e9-e8fb2ca32ec5 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:39.026472 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:39.027410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:39.027697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:39.027976 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] PATH_INFO: `/v3/users/5313f89806b94c2080a3ec1f461c1f3a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:39.031078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5313f89806b94c2080a3ec1f461c1f3a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:39.031432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:39.164254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5313f89806b94c2080a3ec1f461c1f3a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:39.194374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5313f89806b94c2080a3ec1f461c1f3a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:39.195692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5313f89806b94c2080a3ec1f461c1f3a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:39.214018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5313f89806b94c2080a3ec1f461c1f3a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:39.215590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d2ca03ca-a86b-4885-9e6f-30ac646dd300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5313f89806b94c2080a3ec1f461c1f3a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:39.250490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1074/2138] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:38 2022] DELETE /identity/v3/users/5313f89806b94c2080a3ec1f461c1f3a => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:40.349604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:40.352120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:40.384480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978dd63-678c-452e-ba57-20d1568d587e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:40.608937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:40.610306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:40.611771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:40.612197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] PATH_INFO: `/v3/projects/52d7a90da0924639b7c7fcd06dcbd058` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:40.614360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d7a90da0924639b7c7fcd06dcbd058)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:40.618165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:40.756128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d7a90da0924639b7c7fcd06dcbd058', 'request_id': 'req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:40.772847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d7a90da0924639b7c7fcd06dcbd058', 'request_id': 'req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:40.782453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d7a90da0924639b7c7fcd06dcbd058', 'request_id': 'req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:40.800122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:40.806888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] The token cache is being invalidate because project 52d7a90da0924639b7c7fcd06dcbd058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:40.816443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:40.817769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0d0a4115-e1fe-4153-b7ef-e4b5b7430f71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:40.825362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1065/2139] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:40 2022] DELETE /identity/v3/projects/52d7a90da0924639b7c7fcd06dcbd058 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:40.825773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d915-5c67-4820-80a2-0091b56abaea None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:40.826256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d915-5c67-4820-80a2-0091b56abaea None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:40.826521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d915-5c67-4820-80a2-0091b56abaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:40.854312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b188d915-5c67-4820-80a2-0091b56abaea None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:40.856671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b188d915-5c67-4820-80a2-0091b56abaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:40.885171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:41.008539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1075/2140] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:41.022116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185926-90b8-47ab-a047-11ad92beb6a4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.022475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185926-90b8-47ab-a047-11ad92beb6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.022912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185926-90b8-47ab-a047-11ad92beb6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.075936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ae185926-90b8-47ab-a047-11ad92beb6a4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:41.078795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ae185926-90b8-47ab-a047-11ad92beb6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:41.113155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1066/2141] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:41.115672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188d915-5c67-4820-80a2-0091b56abaea None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:41.144408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188d915-5c67-4820-80a2-0091b56abaea None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:41.154512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:41.155250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.155837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.156357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.157885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:41.158463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:41.164026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:41.197215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1076/2142] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:41.203616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185926-90b8-47ab-a047-11ad92beb6a4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:41.215365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185926-90b8-47ab-a047-11ad92beb6a4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:41.223861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:41.225098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.225098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.225336 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.225994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:41.228000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:41.273011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1067/2143] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:41 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:41.279351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:41.298492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb4017-b9a7-4e36-8db1-5cfe2b8a0c61 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:41.437877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:41.439106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.439445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.445031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.446537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:41.456776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:41.502541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1077/2144] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:41 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:41.508556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:41.531802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c465-5263-4a68-98ca-cf34454418cf admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:41.541209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:41.543052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.543779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.544315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.545928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:41.549239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:41.579713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1068/2145] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:41.580968 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:41.607513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2200be-ecb9-475d-99d7-ee8c77502a1e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:41.616929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60361a32-9a06-417f-a5b0-21103e491021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:41.618479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60361a32-9a06-417f-a5b0-21103e491021 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.618986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60361a32-9a06-417f-a5b0-21103e491021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.619479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60361a32-9a06-417f-a5b0-21103e491021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.621460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60361a32-9a06-417f-a5b0-21103e491021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:41.624882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60361a32-9a06-417f-a5b0-21103e491021 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:41.659464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1078/2146] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:04:41.666194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:41.696218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438bbb4-fdea-4c48-8eef-ddb321ca8c7f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:41.706695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:41.708963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.709831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.710564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] PATH_INFO: `/v3/projects/2ea1b44e421d45389c93f3088d61755d/users/d7405c23dd134097a3a8c2da44270566/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.769350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea1b44e421d45389c93f3088d61755d, user_id=d7405c23dd134097a3a8c2da44270566, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:41.773110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:41.821266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1069/2147] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:41 2022] PUT /identity/v3/projects/2ea1b44e421d45389c93f3088d61755d/users/d7405c23dd134097a3a8c2da44270566/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:41.825986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a805a35-d93d-4068-b3d2-9ba925616800 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:41.825986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a805a35-d93d-4068-b3d2-9ba925616800 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:41.825986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a805a35-d93d-4068-b3d2-9ba925616800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:41.873855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3a805a35-d93d-4068-b3d2-9ba925616800 None None] MFA Rules not processed for user `d7405c23dd134097a3a8c2da44270566`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:41.875466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3a805a35-d93d-4068-b3d2-9ba925616800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:42.017306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:42.020594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1079/2148] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:42.160351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1a70f-5427-48e8-85b4-52e6e1df6ddc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:42.172220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:42.173744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:42.174246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:42.174750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:42.179335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:42.183239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:42.280923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1070/2149] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:42 2022] POST /identity/v3/projects => generated 373 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:42.286495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a805a35-d93d-4068-b3d2-9ba925616800 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:42.306864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a805a35-d93d-4068-b3d2-9ba925616800 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:42.355765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:42.486601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:42.489955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:42.517134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:42.518237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:42.518700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:42.519174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:42.520510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:42.524175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:42.576532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1080/2150] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:42 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:42.581881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:42.605649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08428d3d-418d-453e-ae1f-269768975e12 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:42.615634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:42.616887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:42.617602 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:42.618252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:42.621369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:42.625127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:42.669260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1081/2151] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:42.671548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:42.702183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b70b5-1a23-413b-8341-75774406edce admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:42.708810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:42.710218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:42.710739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:42.711197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] PATH_INFO: `/v3/users/719204a59b4c4344adb0d1b4c698003e` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:42.712933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=719204a59b4c4344adb0d1b4c698003e)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:42.716200 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:42.725829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865b591-b289-42b0-8d6c-6887fa9e36c4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:42.740087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:42.746224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:42.748220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:42.748764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] PATH_INFO: `/v3/projects/edb08f0eeaa1481d8354feed3cfb7cae/users/85d33198f26840a0826a3d20306a8718/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:42.759795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:42.790014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb08f0eeaa1481d8354feed3cfb7cae, user_id=85d33198f26840a0826a3d20306a8718, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:42.793934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:42.815302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1082/2152] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:42 2022] PUT /identity/v3/projects/edb08f0eeaa1481d8354feed3cfb7cae/users/85d33198f26840a0826a3d20306a8718/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:42.821737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:42.823023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719204a59b4c4344adb0d1b4c698003e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:42.826524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719204a59b4c4344adb0d1b4c698003e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:42.853403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719204a59b4c4344adb0d1b4c698003e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:42.854188 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719204a59b4c4344adb0d1b4c698003e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:42.891847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d04b2354-8ca5-4b57-83b2-dbacefc5de45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719204a59b4c4344adb0d1b4c698003e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:42.923289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1071/2153] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:42 2022] DELETE /identity/v3/users/719204a59b4c4344adb0d1b4c698003e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:42.929180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:42.929601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:42.929950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:42.952365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f6e4d-31cd-492d-9012-0608f9139d90 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:42.957258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:42.959947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:43.049856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:43.051252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:43.052467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:43.053195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:43.054416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:43.056370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:43.057395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1072/2154] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:43.078179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1083/2155] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:43.079393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:43.081589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:43.082128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:43.086221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:43.136031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0a4cc-630f-4324-8bb0-4fd6036d303d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:43.144756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061979b4-f9be-400b-9df7-fbbcbb2a3f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:43.148702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061979b4-f9be-400b-9df7-fbbcbb2a3f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:43.149373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061979b4-f9be-400b-9df7-fbbcbb2a3f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:43.150151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061979b4-f9be-400b-9df7-fbbcbb2a3f25 admin admin] PATH_INFO: `/v3/projects/edb08f0eeaa1481d8354feed3cfb7cae/users/85d33198f26840a0826a3d20306a8718/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:43.158794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061979b4-f9be-400b-9df7-fbbcbb2a3f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb08f0eeaa1481d8354feed3cfb7cae, user_id=85d33198f26840a0826a3d20306a8718, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:43.166053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061979b4-f9be-400b-9df7-fbbcbb2a3f25 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:43.206097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1084/2156] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:43 2022] PUT /identity/v3/projects/edb08f0eeaa1481d8354feed3cfb7cae/users/85d33198f26840a0826a3d20306a8718/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:43.217309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:43.218908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:43.219165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:43.224417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934dada-8292-42b3-858f-6f36ebe1c97c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:43.227371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:43.263218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] MFA Rules not processed for user `85d33198f26840a0826a3d20306a8718`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:43.264661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:43.299866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baeebe7-9e3e-4de1-a9d7-3ec32d90e069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:43.306440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeebe7-9e3e-4de1-a9d7-3ec32d90e069 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:43.308776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeebe7-9e3e-4de1-a9d7-3ec32d90e069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:43.309416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeebe7-9e3e-4de1-a9d7-3ec32d90e069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:43.324526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:43.376668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1085/2157] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:43.391632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:43.393873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:43.425903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baeebe7-9e3e-4de1-a9d7-3ec32d90e069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:43.427675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baeebe7-9e3e-4de1-a9d7-3ec32d90e069 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:43.449166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1073/2158] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:43.489706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4eb27-7fae-49ee-8b11-bc98d9fa5705 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:43.506847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:43.508848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:43.508848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:43.508848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:43.513058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:43.630567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:43.635044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:43.662498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1086/2159] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:43.915563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9ee96-0437-45ad-a1b0-bbce71b999d3 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:43.916352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9ee96-0437-45ad-a1b0-bbce71b999d3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:43.916352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9ee96-0437-45ad-a1b0-bbce71b999d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:43.953157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-14e9ee96-0437-45ad-a1b0-bbce71b999d3 None None] MFA Rules not processed for user `6663857e60ca4440a9a620007be4c2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:43.954638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-14e9ee96-0437-45ad-a1b0-bbce71b999d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:44.048521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1074/2160] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:44.054186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.055521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:44.071635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b1d45-7a0b-4b52-94dc-3af48109c5cd service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:44.078998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:44.079818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.080219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.080586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.081511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:44.084284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:44.165770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1087/2161] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:44 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:44.169063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9ee96-0437-45ad-a1b0-bbce71b999d3 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.188873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.202017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:44.227371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9ee96-0437-45ad-a1b0-bbce71b999d3 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:44.300250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:44.371863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8558-fe11-47be-ad35-ffb54988d7a8 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:44.377483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:44.378405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.378835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.378993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] PATH_INFO: `/v3/projects/69e2bbfe423846dc9755f233e89ecc8f` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.380565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e2bbfe423846dc9755f233e89ecc8f)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:44.383096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:44.465470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:44.466266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.466563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.466848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.467797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:44.472570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:44.484247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e2bbfe423846dc9755f233e89ecc8f', 'request_id': 'req-34fc6173-aa07-4b2e-9375-2678d9f20f4b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:44.494199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e2bbfe423846dc9755f233e89ecc8f', 'request_id': 'req-34fc6173-aa07-4b2e-9375-2678d9f20f4b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:44.511306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e2bbfe423846dc9755f233e89ecc8f', 'request_id': 'req-34fc6173-aa07-4b2e-9375-2678d9f20f4b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:44.533323 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1075/2162] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:44 2022] POST /identity/v3/users => generated 298 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:44.534763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.549993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] The token cache is being invalidate because project 69e2bbfe423846dc9755f233e89ecc8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:44.552213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:44.554265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-34fc6173-aa07-4b2e-9375-2678d9f20f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:44.560913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1088/2163] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:44 2022] DELETE /identity/v3/projects/69e2bbfe423846dc9755f233e89ecc8f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:44.562151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec3a67-6279-450e-a638-ce92eb22cf73 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:44.568773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b1f71-0b5d-4143-8c79-c9fd39a3c653 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.569847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b1f71-0b5d-4143-8c79-c9fd39a3c653 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.570432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b1f71-0b5d-4143-8c79-c9fd39a3c653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.610711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:44.619234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-927b1f71-0b5d-4143-8c79-c9fd39a3c653 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:44.620896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-927b1f71-0b5d-4143-8c79-c9fd39a3c653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:44.677876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:44.679124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.680441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.680964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.682434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:44.685644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:44.711106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1076/2164] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:44.717806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.727186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:44.728123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1089/2165] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:44.736371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c6a5c-045c-4e39-9de6-6b9728b1c757 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.736640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c6a5c-045c-4e39-9de6-6b9728b1c757 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.736930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c6a5c-045c-4e39-9de6-6b9728b1c757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.758422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365542c7-b16a-4b45-a3e6-ed1f109a5999 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:44.764202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-cc3c6a5c-045c-4e39-9de6-6b9728b1c757 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:44.765845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-cc3c6a5c-045c-4e39-9de6-6b9728b1c757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:44.771045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:44.772250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.772681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.773417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] PATH_INFO: `/v3/projects/02814cee7f27425b848a6dfc38e5a714/users/35b8945cc3014361af5fc2c7f92db8a5/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.788940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1090/2166] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:44.789981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c6a5c-045c-4e39-9de6-6b9728b1c757 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.802580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c6a5c-045c-4e39-9de6-6b9728b1c757 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:44.806259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02814cee7f27425b848a6dfc38e5a714, user_id=35b8945cc3014361af5fc2c7f92db8a5, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:44.810571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:44.811554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.811996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.812378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:44.812586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.813553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:44.814051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:44.816237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:44.836836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1091/2167] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:44.839055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.853061 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9135f4-f103-4fa3-b3d3-b4d78ae491b5 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:44.868009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1077/2168] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:44 2022] PUT /identity/v3/projects/02814cee7f27425b848a6dfc38e5a714/users/35b8945cc3014361af5fc2c7f92db8a5/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:44.875333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:44.900924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:44.981474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:44.982719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:44.983267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:44.983988 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:44.986069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:44.990398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:45.007719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafb8d-0457-4342-8592-616c25e1af63 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:45.019232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:45.022514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:45.023085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:45.023404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:45.025623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:45.027031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:45.042706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1078/2169] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:45.047431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1092/2170] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:44 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:45.047979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:45.048649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:45.061041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741731e2-a316-47ae-9c73-f89c32995616 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:45.137130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acd2d5-bded-47f5-b182-27b057a3d899 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:45.143468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:45.144173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:45.144458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:45.144782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:45.146060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:45.146799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:45.147464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:45.147907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:45.148211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] PATH_INFO: `/v3/projects/02814cee7f27425b848a6dfc38e5a714/users/35b8945cc3014361af5fc2c7f92db8a5/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:45.148798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:45.150640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02814cee7f27425b848a6dfc38e5a714, user_id=35b8945cc3014361af5fc2c7f92db8a5, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:45.153531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:45.166028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1079/2171] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:45 2022] PUT /identity/v3/projects/02814cee7f27425b848a6dfc38e5a714/users/35b8945cc3014361af5fc2c7f92db8a5/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:45.175255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1093/2172] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:45 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:45.181121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:45.252486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:45.320271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d1f82-4c99-4e8f-bfd3-8b2fd0a09a44 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:45.338673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:45.339819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:45.340426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:45.340920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:45.342824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:45.349832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:45.385391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1080/2173] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:45.391595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:45.424919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c103d82-4dd9-47f0-afc7-f345087a3ae3 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:45.441209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec54221-6071-42a9-9722-ccb8882448e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:45.443485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54221-6071-42a9-9722-ccb8882448e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:45.444225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54221-6071-42a9-9722-ccb8882448e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:45.445292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54221-6071-42a9-9722-ccb8882448e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:45.449476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec54221-6071-42a9-9722-ccb8882448e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:45.455195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec54221-6071-42a9-9722-ccb8882448e4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:45.500243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1094/2174] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:45.504704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:45.517550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce24d-51d8-4e3f-9751-8c472cc2a455 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:45.523246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:45.524342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:45.524632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:45.524908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] PATH_INFO: `/v3/projects/d8864e0889984983bbe1ace5f6f51705/users/f19181a282a742a98f1781e61b3bce8d/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:45.552791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8864e0889984983bbe1ace5f6f51705, user_id=f19181a282a742a98f1781e61b3bce8d, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:45.556541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:45.611896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1081/2175] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:45 2022] PUT /identity/v3/projects/d8864e0889984983bbe1ace5f6f51705/users/f19181a282a742a98f1781e61b3bce8d/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:45.625659 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:45.625930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:45.626300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:45.677612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] MFA Rules not processed for user `f19181a282a742a98f1781e61b3bce8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:45.678904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:45.835991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1095/2176] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:45.850790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:45.853933 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:46.093914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d419-d375-4be8-8c89-536e68a43549 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:46.114303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3684aaf-d0c6-493e-a92c-2c30a7bfde37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:46.115444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3684aaf-d0c6-493e-a92c-2c30a7bfde37 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:46.115947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3684aaf-d0c6-493e-a92c-2c30a7bfde37 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:46.116450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3684aaf-d0c6-493e-a92c-2c30a7bfde37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:46.138019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3684aaf-d0c6-493e-a92c-2c30a7bfde37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:46.142227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3684aaf-d0c6-493e-a92c-2c30a7bfde37 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:46.163719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1082/2177] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:48.137916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:48.142063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:48.259589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:48.346578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf808f19-39a6-40a7-8085-2374d51723d4 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:48.355443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:48.357501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:48.358031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:48.358530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] PATH_INFO: `/v3/users/d7405c23dd134097a3a8c2da44270566` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:48.361416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7405c23dd134097a3a8c2da44270566)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:48.366151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:48.589106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7405c23dd134097a3a8c2da44270566'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:48.626465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7405c23dd134097a3a8c2da44270566'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:48.627823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7405c23dd134097a3a8c2da44270566'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:48.647134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7405c23dd134097a3a8c2da44270566'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:48.649503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7405c23dd134097a3a8c2da44270566'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:48.685380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1096/2178] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:48 2022] DELETE /identity/v3/users/d7405c23dd134097a3a8c2da44270566 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:48.691232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:48.693843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:48.693843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:48.772050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:48.778637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:48.966019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1083/2179] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:48.976451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:48.980294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:49.226031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a645d39-e950-48dd-8d09-04165d0961f2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:49.231636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:49.232433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:49.232769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:49.233171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:49.256662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:49.261449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:49.286024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1097/2180] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:49.944840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:49.947907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:49.979277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b18784-3284-418a-bafb-53341af3f4a2 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:49.986775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:49.988322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:49.988832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:49.989295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] PATH_INFO: `/v3/projects/2ea1b44e421d45389c93f3088d61755d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:49.990891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea1b44e421d45389c93f3088d61755d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:49.993849 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:50.027397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:50.074149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea1b44e421d45389c93f3088d61755d', 'request_id': 'req-cee48b89-72f9-46f3-a711-55a519eeb71e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.093671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea1b44e421d45389c93f3088d61755d', 'request_id': 'req-cee48b89-72f9-46f3-a711-55a519eeb71e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.106654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea1b44e421d45389c93f3088d61755d', 'request_id': 'req-cee48b89-72f9-46f3-a711-55a519eeb71e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.144485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] The token cache is being invalidate because project 2ea1b44e421d45389c93f3088d61755d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:50.144955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.161568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.164959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1084/2181] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:49 2022] DELETE /identity/v3/projects/2ea1b44e421d45389c93f3088d61755d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:50.170666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:50.195336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5ead6-9bb4-4729-b7de-d7c73acc8acb service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:50.250382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:50.251912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:50.342038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:50.378057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:50.380811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:50.381341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:50.381915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] PATH_INFO: `/v3/users/85d33198f26840a0826a3d20306a8718` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:50.385438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d33198f26840a0826a3d20306a8718)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:50.390112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:50.393273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48b89-72f9-46f3-a711-55a519eeb71e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:50.401545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:50.403081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:50.403600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:50.404060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] PATH_INFO: `/v3/users/f19181a282a742a98f1781e61b3bce8d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:50.406817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19181a282a742a98f1781e61b3bce8d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:50.412127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:50.617981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19181a282a742a98f1781e61b3bce8d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.625541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19181a282a742a98f1781e61b3bce8d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.654800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19181a282a742a98f1781e61b3bce8d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.656532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19181a282a742a98f1781e61b3bce8d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.680048 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d33198f26840a0826a3d20306a8718'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.715186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d33198f26840a0826a3d20306a8718'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.717186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d33198f26840a0826a3d20306a8718'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.734260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-19207dd2-de31-438a-a7a1-af3f0d3847e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19181a282a742a98f1781e61b3bce8d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.761450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d33198f26840a0826a3d20306a8718'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.763199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1085/2182] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:50 2022] DELETE /identity/v3/users/f19181a282a742a98f1781e61b3bce8d => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:50.769133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d33198f26840a0826a3d20306a8718'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:50.770658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:50.770949 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:50.771235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:50.807599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1098/2183] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:50 2022] DELETE /identity/v3/users/85d33198f26840a0826a3d20306a8718 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:50.827673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:50.829018 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:50.889515 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:51.094143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1086/2184] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:51.124460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:51.126720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:51.337122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1ed98-b5ec-4b00-a3cd-4057ddcc4d9b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:51.348844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:51.352467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:51.353249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:51.354040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:51.374447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:51.375063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:51.391316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1099/2185] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:51.429356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:51.432383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:51.490913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1c586-26c3-44f2-85bd-9f0a44f1a353 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:51.519821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:51.522995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:51.524824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:51.525116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] PATH_INFO: `/v3/users/88428b2172c24e13a6e0fbb5e70e738d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:51.528873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88428b2172c24e13a6e0fbb5e70e738d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:51.539482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:51.651623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:51.654352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:51.721722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6c872-133d-44ff-9826-d5c2857f3b77 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:51.734933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:51.735555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88428b2172c24e13a6e0fbb5e70e738d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.735982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:51.736426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:51.736863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] PATH_INFO: `/v3/projects/edb08f0eeaa1481d8354feed3cfb7cae` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:51.737101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88428b2172c24e13a6e0fbb5e70e738d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.738138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb08f0eeaa1481d8354feed3cfb7cae)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:51.749856 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:51.759991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88428b2172c24e13a6e0fbb5e70e738d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.760918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88428b2172c24e13a6e0fbb5e70e738d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.776390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-dd70fa10-5e81-4942-b664-c410964aece5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88428b2172c24e13a6e0fbb5e70e738d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.821621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1087/2186] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:51 2022] DELETE /identity/v3/users/88428b2172c24e13a6e0fbb5e70e738d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:51.827069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95dd1d-9411-4ff9-83b9-7d2a00f3e060 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:51.827663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95dd1d-9411-4ff9-83b9-7d2a00f3e060 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:51.827835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95dd1d-9411-4ff9-83b9-7d2a00f3e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:51.858893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-fd95dd1d-9411-4ff9-83b9-7d2a00f3e060 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:51.861960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fd95dd1d-9411-4ff9-83b9-7d2a00f3e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:51.887378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb08f0eeaa1481d8354feed3cfb7cae', 'request_id': 'req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.903230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb08f0eeaa1481d8354feed3cfb7cae', 'request_id': 'req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.909612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:51.934184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb08f0eeaa1481d8354feed3cfb7cae', 'request_id': 'req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.959492 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] The token cache is being invalidate because project edb08f0eeaa1481d8354feed3cfb7cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:51.959935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.964343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8a0c3e2b-52f5-4159-8165-c3f596ba7b90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:51.967930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1100/2187] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:51 2022] DELETE /identity/v3/projects/edb08f0eeaa1481d8354feed3cfb7cae => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:51.972267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:51.972658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:51.974269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.008236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1088/2188] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:52.016744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:52.018741 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:52.045277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae57740-9201-4ef9-a300-b8f47d74caeb None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.045823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae57740-9201-4ef9-a300-b8f47d74caeb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.046272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae57740-9201-4ef9-a300-b8f47d74caeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.046812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1101/2189] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:52.057064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:52.059270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:52.089297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9ae57740-9201-4ef9-a300-b8f47d74caeb None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:52.092795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9ae57740-9201-4ef9-a300-b8f47d74caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:52.121465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1089/2190] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:04:52.126098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae57740-9201-4ef9-a300-b8f47d74caeb None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:52.158966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae57740-9201-4ef9-a300-b8f47d74caeb None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:52.181569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:52.183082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.183457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.183776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.184797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:52.185024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:52.194886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:52.213688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99791aa8-446d-4651-90d1-08e0134fd467 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:52.222959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:52.223735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.224186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.224458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.234010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1090/2191] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:52.236001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:52.243262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:52.244920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:52.264679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1102/2192] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:52.271481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9c990-82f2-4677-9f42-36e70cb847ba admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:52.286279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:52.287535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.288199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.288764 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.289858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:52.293030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:52.349265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:52.377581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1091/2193] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:52 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:52.382481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:52.400367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4fe20-d65e-4b5a-b264-e183e2d26d97 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:52.608822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:52.609672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.609870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.610149 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.619369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:52.621849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:52.693057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1103/2194] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:52 2022] POST /identity/v3/users => generated 288 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:52.701270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:52.741276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931762c-70e1-483f-9fe8-3716e50a7bcb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:52.764806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:52.766169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.766680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.767208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.773315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:52.776500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:52.839402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1092/2195] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:52.844352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:52.878672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a59c49-33a7-48f2-a41c-110c1a0617c3 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:52.897374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae022dd-4fe3-43b6-8192-b085047c11a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:52.898715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae022dd-4fe3-43b6-8192-b085047c11a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.899166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae022dd-4fe3-43b6-8192-b085047c11a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.899993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae022dd-4fe3-43b6-8192-b085047c11a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:52.902665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae022dd-4fe3-43b6-8192-b085047c11a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:52.905420 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae022dd-4fe3-43b6-8192-b085047c11a9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:52.932866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1104/2196] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:52.937464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:52.960359 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247c78c-ef04-48a5-be93-aadd9391d7d7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:52.970861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:52.972438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:52.973718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:52.974055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] PATH_INFO: `/v3/projects/90468df9a8d84e4dac4dc8ff20f1a7d1/users/d437c037d424443d845edc4adf450412/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:53.010065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90468df9a8d84e4dac4dc8ff20f1a7d1, user_id=d437c037d424443d845edc4adf450412, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:53.015939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:53.053793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1093/2197] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:52 2022] PUT /identity/v3/projects/90468df9a8d84e4dac4dc8ff20f1a7d1/users/d437c037d424443d845edc4adf450412/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:53.060889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:53.061408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:53.061883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:53.117215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] MFA Rules not processed for user `d437c037d424443d845edc4adf450412`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:53.118962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:53.181722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:53.186220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:53.207836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:53.244337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1105/2198] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:53.264772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:53.266450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:53.391664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95923a-4eb0-416d-8995-d99bf75a3202 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:53.405986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:53.407506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:53.407506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:53.407506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] PATH_INFO: `/v3/projects/d8864e0889984983bbe1ace5f6f51705` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:53.409023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8864e0889984983bbe1ace5f6f51705)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:53.414037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1f805-d7c8-46c1-92b9-fb7ad54f9fd7 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:53.415579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:53.424574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5363b-94bd-4d27-af43-b39fb542c6e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:53.425279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5363b-94bd-4d27-af43-b39fb542c6e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:53.425949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5363b-94bd-4d27-af43-b39fb542c6e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:53.426116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5363b-94bd-4d27-af43-b39fb542c6e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:53.456341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5363b-94bd-4d27-af43-b39fb542c6e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:53.458407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5363b-94bd-4d27-af43-b39fb542c6e3 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:53.490198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1106/2199] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:53.533677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8864e0889984983bbe1ace5f6f51705', 'request_id': 'req-204cf3db-1147-4c6c-bea6-75bba0e589a6'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.562692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8864e0889984983bbe1ace5f6f51705', 'request_id': 'req-204cf3db-1147-4c6c-bea6-75bba0e589a6'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.577238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8864e0889984983bbe1ace5f6f51705', 'request_id': 'req-204cf3db-1147-4c6c-bea6-75bba0e589a6'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.606738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] The token cache is being invalidate because project d8864e0889984983bbe1ace5f6f51705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:53.607045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.614261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.623028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1094/2200] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:53 2022] DELETE /identity/v3/projects/d8864e0889984983bbe1ace5f6f51705 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:53.626356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc431f-e6fb-46ee-be52-a7bc351e8bbe None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:53.627178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc431f-e6fb-46ee-be52-a7bc351e8bbe None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:53.627915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc431f-e6fb-46ee-be52-a7bc351e8bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:53.696274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:53.698121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:53.733142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-38dc431f-e6fb-46ee-be52-a7bc351e8bbe None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:53.741054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-38dc431f-e6fb-46ee-be52-a7bc351e8bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:53.802988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:53.810112 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204cf3db-1147-4c6c-bea6-75bba0e589a6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:53.817405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:53.818715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:53.819141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:53.819433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] PATH_INFO: `/v3/projects/de8c46aaa09443f485ac01c23c548554` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:53.821415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8c46aaa09443f485ac01c23c548554)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:53.824797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:53.901449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8c46aaa09443f485ac01c23c548554', 'request_id': 'req-ca17d962-a648-43f1-a071-38f381b06c7d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.908842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8c46aaa09443f485ac01c23c548554', 'request_id': 'req-ca17d962-a648-43f1-a071-38f381b06c7d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.936068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8c46aaa09443f485ac01c23c548554', 'request_id': 'req-ca17d962-a648-43f1-a071-38f381b06c7d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.972494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] The token cache is being invalidate because project de8c46aaa09443f485ac01c23c548554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:04:53.974679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.977042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ca17d962-a648-43f1-a071-38f381b06c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:53.977380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1107/2201] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:53.980397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1095/2202] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:04:53 2022] DELETE /identity/v3/projects/de8c46aaa09443f485ac01c23c548554 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:53.989931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea30f3-cf7f-4e14-a7c7-fcb59fb3de4d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:53.989931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea30f3-cf7f-4e14-a7c7-fcb59fb3de4d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:53.989931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea30f3-cf7f-4e14-a7c7-fcb59fb3de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:53.991375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b8996-a546-4f2c-8648-4c11d188874a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:53.992124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b8996-a546-4f2c-8648-4c11d188874a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:53.992825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b8996-a546-4f2c-8648-4c11d188874a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:54.042734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-80ea30f3-cf7f-4e14-a7c7-fcb59fb3de4d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:54.051963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-80ea30f3-cf7f-4e14-a7c7-fcb59fb3de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:54.071312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-274b8996-a546-4f2c-8648-4c11d188874a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:54.074416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-274b8996-a546-4f2c-8648-4c11d188874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:54.203591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1108/2203] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:54.212955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1af7-84dd-449f-a141-44c0b4eae1fb None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:54.213524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1af7-84dd-449f-a141-44c0b4eae1fb None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:54.213904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1af7-84dd-449f-a141-44c0b4eae1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:54.261540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-799a1af7-84dd-449f-a141-44c0b4eae1fb None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:54.265177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-799a1af7-84dd-449f-a141-44c0b4eae1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:54.287842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1109/2204] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:54.294851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a1af7-84dd-449f-a141-44c0b4eae1fb None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:54.307644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1096/2205] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:54.321533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b8996-a546-4f2c-8648-4c11d188874a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:54.324263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a1af7-84dd-449f-a141-44c0b4eae1fb None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:54.337013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:54.340681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:54.341146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:54.343327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:54.344422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:54.347269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:54.353418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:54.368903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b8996-a546-4f2c-8648-4c11d188874a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:54.374348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:54.375573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:54.376070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:54.376574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:54.379919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:04:54.380477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:54.384930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:54.423878 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1097/2206] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:04:54.431348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:54.440057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1110/2207] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:04:54.447855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:54.490252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df624ed5-4de5-43fa-a9a3-a555724638b7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:54.499681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf8188-1053-40a0-9039-50cd234f6314 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:54.510224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:54.510929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:54.511218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:54.511500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:54.516596 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:54.522787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:54.523165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:54.524346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:54.524924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:54.529498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:54.530919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:54.540036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:54.580606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1098/2208] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:54 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:54.588542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:54.622716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:54.638931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1111/2209] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:04:54 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:04:54.645813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:54.693548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:54.721831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156950f-e5da-45a1-bfce-853825408056 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:54.928140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c04b-5a8c-4550-afd1-05d73c261b53 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:54.963062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:54.967148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:54.967880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:54.969478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:54.974381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:54.982135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.032223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1112/2210] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:54 2022] POST /identity/v3/users => generated 287 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:55.038694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:55.057881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb29302-e571-4f89-9784-f34678bea899 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:55.067827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:55.068539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.068825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.069221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.069991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:55.071736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.079916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:55.085665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.086291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.086822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.088237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:55.101359 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.120651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1113/2211] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:55.126948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:55.168105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160a47b-1041-4c46-9b42-3516168954b7 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:55.190084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:55.194062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1099/2212] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:04:54 2022] POST /identity/v3/users => generated 303 bytes in 607 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:04:55.197897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.198171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.198467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.198660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:55.199250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:55.208771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.248821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1114/2213] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:04:55.251194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c668286-8d73-47af-8f61-6189fe05bf8c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:55.256108 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:55.276788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785fd3-c53f-4bc2-8d02-ffbfec7b33ce admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:55.279409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:55.282420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.283054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.283909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.290658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43b282-16d5-4b7d-87b5-dcc3f928e6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:55.292741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:55.303738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b282-16d5-4b7d-87b5-dcc3f928e6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.303738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b282-16d5-4b7d-87b5-dcc3f928e6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.303738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b282-16d5-4b7d-87b5-dcc3f928e6e7 admin admin] PATH_INFO: `/v3/projects/26a20f6f9f254aae91c952ee44a4ff74/users/de6636a0522642799e43a8f02ee602ea/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.304125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.335332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43b282-16d5-4b7d-87b5-dcc3f928e6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a20f6f9f254aae91c952ee44a4ff74, user_id=de6636a0522642799e43a8f02ee602ea, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:55.338294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43b282-16d5-4b7d-87b5-dcc3f928e6e7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.342249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1100/2214] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:04:55.350674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:55.366552 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1115/2215] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:55 2022] PUT /identity/v3/projects/26a20f6f9f254aae91c952ee44a4ff74/users/de6636a0522642799e43a8f02ee602ea/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:04:55.375532 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.377113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.377446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.390580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d9cfa-3602-4b19-aabc-918fd6a14281 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:55.445051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:55.445834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:55.653274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:55.654431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.654955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.658368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.662247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:55.666069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.677098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1116/2216] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:55.691110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1101/2217] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:04:55.698971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:55.699245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:55.702855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:55.720769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb774b-9ffd-4902-9821-f21ffdbbef36 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:55.736035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47de43-7428-4341-b5c6-e50838c00fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:55.738825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47de43-7428-4341-b5c6-e50838c00fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.739477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47de43-7428-4341-b5c6-e50838c00fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.740207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47de43-7428-4341-b5c6-e50838c00fe7 admin admin] PATH_INFO: `/v3/projects/7ff2f23d907640329a9468e514d8978a/users/d39918f5dad14060a8c659c3b3fd40dd/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.779961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47de43-7428-4341-b5c6-e50838c00fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff2f23d907640329a9468e514d8978a, user_id=d39918f5dad14060a8c659c3b3fd40dd, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:55.786051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47de43-7428-4341-b5c6-e50838c00fe7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:55.826822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1102/2218] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:04:55 2022] PUT /identity/v3/projects/7ff2f23d907640329a9468e514d8978a/users/d39918f5dad14060a8c659c3b3fd40dd/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:55.837177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2c50d-4a9e-4c55-8349-92bab1de0af9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:55.839221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2c50d-4a9e-4c55-8349-92bab1de0af9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:55.839771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2c50d-4a9e-4c55-8349-92bab1de0af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:55.889806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:55.945090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e57044-067d-4d90-bf11-02cddd5749ce None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:55.954377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-fdf2c50d-4a9e-4c55-8349-92bab1de0af9 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:55.961930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fdf2c50d-4a9e-4c55-8349-92bab1de0af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:56.203157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:04:56.275194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:56.277585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:56.279308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1103/2219] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:56.281893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:56.283299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:56.291943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2c50d-4a9e-4c55-8349-92bab1de0af9 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:56.322461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2c50d-4a9e-4c55-8349-92bab1de0af9 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:56.323733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:56.328007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:56.332676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4969aad-614e-4fce-82e3-55e4534a884c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:56.334533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4969aad-614e-4fce-82e3-55e4534a884c service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:56.335550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4969aad-614e-4fce-82e3-55e4534a884c service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:56.336249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4969aad-614e-4fce-82e3-55e4534a884c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:56.352728 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4969aad-614e-4fce-82e3-55e4534a884c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:56.354669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4969aad-614e-4fce-82e3-55e4534a884c service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:56.382550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1117/2220] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:04:56.383866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1104/2221] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:04:58.815781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:58.819454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:58.840554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cbb53-18c8-40be-b749-b40c390bfc33 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:58.859140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:58.861438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:58.865210 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:58.865722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] PATH_INFO: `/v3/users/d437c037d424443d845edc4adf450412` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:58.869605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d437c037d424443d845edc4adf450412)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:58.877649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:59.072653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd437c037d424443d845edc4adf450412'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:59.125631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd437c037d424443d845edc4adf450412'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:59.126927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd437c037d424443d845edc4adf450412'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:59.156424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd437c037d424443d845edc4adf450412'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:59.157423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd437c037d424443d845edc4adf450412'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:04:59.207142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1118/2222] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:04:58 2022] DELETE /identity/v3/users/d437c037d424443d845edc4adf450412 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:04:59.214430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0d76-2dac-4799-a7ae-aee8e8dde69d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:59.215187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0d76-2dac-4799-a7ae-aee8e8dde69d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:59.215418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0d76-2dac-4799-a7ae-aee8e8dde69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:59.256700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3a7b0d76-2dac-4799-a7ae-aee8e8dde69d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:04:59.259983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3a7b0d76-2dac-4799-a7ae-aee8e8dde69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:59.560745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1105/2223] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:04:59.667383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:04:59.670273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:04:59.812589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23f9f-77c7-4d36-a300-2e7bfd4e0a7d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:04:59.827947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:04:59.827947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:04:59.827947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:04:59.827947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:04:59.849452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:04:59.853736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:04:59.876825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1119/2224] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:05:00.396313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70de992-9c19-4c97-a244-447837723cb0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:00.396870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70de992-9c19-4c97-a244-447837723cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:00.397265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70de992-9c19-4c97-a244-447837723cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:00.506800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-f70de992-9c19-4c97-a244-447837723cb0 None None] MFA Rules not processed for user `d39918f5dad14060a8c659c3b3fd40dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:00.508133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f70de992-9c19-4c97-a244-447837723cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:00.663889 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1106/2225] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:00.670583 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:00.672881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:00.705279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d345060-71ec-4c93-b9cc-c618028a866e service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:00.719518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:00.721461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:00.723188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:00.725412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:00.728351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:00.731952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:00.752840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:00.792792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1120/2226] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:05:00 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:00.796637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70de992-9c19-4c97-a244-447837723cb0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:00.815163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70de992-9c19-4c97-a244-447837723cb0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:00.955201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:00.955923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:00.956212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:00.956558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:00.958405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:00.961051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:00.973590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:00.999157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1107/2227] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:05:00 2022] POST /identity/v3/users => generated 309 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:01.005462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:01.030620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674aed49-5c04-404e-bf1e-fbbf33d50517 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:01.047963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:01.049059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:01.049588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:01.050150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:01.052120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:01.056732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:01.073842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1121/2228] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:01.078530 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:01.092102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef185b-b9fb-4b1f-95a1-f7bf057da95c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:01.101298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:01.102441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:01.102737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:01.103021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] PATH_INFO: `/v3/projects/e21e901091484c4cbe5a5bc5ba8034b7/users/8d82b72589ca4b72bac37042aad736cc/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:01.136862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21e901091484c4cbe5a5bc5ba8034b7, user_id=8d82b72589ca4b72bac37042aad736cc, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:01.144588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:01.190565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1108/2229] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:01 2022] PUT /identity/v3/projects/e21e901091484c4cbe5a5bc5ba8034b7/users/8d82b72589ca4b72bac37042aad736cc/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:01.192091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:01.204681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991dda4-cb5d-4a8b-b91b-804942a1093a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:01.344500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:01.346208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:01.347007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:01.348507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:01.350320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:01.354126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:01.375718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1122/2230] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:01.383782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:01.404430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff1e6-5eaf-41e1-abc1-ad5965ced6a4 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:01.414573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523e939-26df-48c5-9de1-38d421ccbe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:01.416644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523e939-26df-48c5-9de1-38d421ccbe26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:01.417502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523e939-26df-48c5-9de1-38d421ccbe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:01.418084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523e939-26df-48c5-9de1-38d421ccbe26 admin admin] PATH_INFO: `/v3/projects/e21e901091484c4cbe5a5bc5ba8034b7/users/8d82b72589ca4b72bac37042aad736cc/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:01.424744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523e939-26df-48c5-9de1-38d421ccbe26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21e901091484c4cbe5a5bc5ba8034b7, user_id=8d82b72589ca4b72bac37042aad736cc, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:01.431288 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523e939-26df-48c5-9de1-38d421ccbe26 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:01.452951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1109/2231] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:01 2022] PUT /identity/v3/projects/e21e901091484c4cbe5a5bc5ba8034b7/users/8d82b72589ca4b72bac37042aad736cc/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:02.004792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:02.007340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:02.039015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:02.039291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:02.039725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:02.173402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] MFA Rules not processed for user `35b8945cc3014361af5fc2c7f92db8a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:02.174446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:02.306872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd8b72-de9d-4ac1-a3d6-71b4edd44416 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:02.449732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1110/2232] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:02.466100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:02.468221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:02.493311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:02.496561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:02.496674 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:02.497027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] PATH_INFO: `/v3/projects/90468df9a8d84e4dac4dc8ff20f1a7d1` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:02.506347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90468df9a8d84e4dac4dc8ff20f1a7d1)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:02.519216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:02.572336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:02.651444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90468df9a8d84e4dac4dc8ff20f1a7d1', 'request_id': 'req-8e7daeaf-7f95-414d-b887-8a76294398a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:02.686220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90468df9a8d84e4dac4dc8ff20f1a7d1', 'request_id': 'req-8e7daeaf-7f95-414d-b887-8a76294398a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:02.702119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258f6d3-1592-4127-abc0-ce5d3e45911f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:02.704490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90468df9a8d84e4dac4dc8ff20f1a7d1', 'request_id': 'req-8e7daeaf-7f95-414d-b887-8a76294398a8'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:02.726511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] The token cache is being invalidate because project 90468df9a8d84e4dac4dc8ff20f1a7d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:05:02.727167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:02.728544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8e7daeaf-7f95-414d-b887-8a76294398a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:02.731392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1123/2233] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:05:02 2022] DELETE /identity/v3/projects/90468df9a8d84e4dac4dc8ff20f1a7d1 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:02.737894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8fcd0-26cf-4a37-a173-137d735fce49 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:02.738165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8fcd0-26cf-4a37-a173-137d735fce49 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:02.738458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8fcd0-26cf-4a37-a173-137d735fce49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:02.814013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:02.884363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:02.885082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:02.885364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:02.885644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:02.895660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:02.935333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-95d8fcd0-26cf-4a37-a173-137d735fce49 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:02.937803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-95d8fcd0-26cf-4a37-a173-137d735fce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:03.265367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1124/2234] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:03.270343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14abb3-ce26-4722-8b88-0a21717f2807 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:03.270909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14abb3-ce26-4722-8b88-0a21717f2807 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:03.271354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14abb3-ce26-4722-8b88-0a21717f2807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:03.294449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:03.297216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:03.369845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1111/2235] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:05:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 902 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:05:03.388839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bf14abb3-ce26-4722-8b88-0a21717f2807 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:03.404645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bf14abb3-ce26-4722-8b88-0a21717f2807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:03.451389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1125/2236] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:03.452450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:03.505997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351a3c2-4b88-4067-a098-8b6a37639b9a service glance] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:03.538426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d873b-9ed7-46f8-bc12-425838be6f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:03.539683 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d873b-9ed7-46f8-bc12-425838be6f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:03.540544 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d873b-9ed7-46f8-bc12-425838be6f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:03.546098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d873b-9ed7-46f8-bc12-425838be6f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:03.547574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d873b-9ed7-46f8-bc12-425838be6f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:05:03.548156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d873b-9ed7-46f8-bc12-425838be6f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:03.555071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d873b-9ed7-46f8-bc12-425838be6f62 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:03.595637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1112/2237] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:05:03.601035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14abb3-ce26-4722-8b88-0a21717f2807 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:03.626353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:03.626992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:03.627485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:03.635248 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14abb3-ce26-4722-8b88-0a21717f2807 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:03.653942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:03.654905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:03.655300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:03.655730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:03.656695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:03.661055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:03.661414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:03.662995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:03.753503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1126/2238] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:03.753503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:03.852570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1113/2239] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:05:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:05:03.936062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:03.937734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:03.997464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:04.011052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213373bc-d766-49eb-a4b3-872ca806396e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:04.046580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:04.048311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:04.049495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:04.052878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:04.054840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:04.064586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:04.135847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f0239-ec1a-46d3-85d4-ec96936e0bc4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:04.186427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1127/2240] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:05:03 2022] POST /identity/v3/users => generated 285 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:04.193328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:04.207894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:04.266831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d077f-8326-4c7a-b0a1-247c7ed2db07 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:04.326453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:04.329793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c5bb2-ca0a-4b66-8fd2-1e99f0cf6d53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:04.330491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c5bb2-ca0a-4b66-8fd2-1e99f0cf6d53 service swift] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:04.330785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c5bb2-ca0a-4b66-8fd2-1e99f0cf6d53 service swift] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:04.331070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c5bb2-ca0a-4b66-8fd2-1e99f0cf6d53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:04.335001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:04.338568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:04.340907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:04.344711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:04.354152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:04.445800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1128/2241] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:04.446945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:04.488708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7b61a-67f6-475f-9ef3-aaba20c58522 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:04.517028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:04.520164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:04.521500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:04.522272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:04.525004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:04.529328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:04.570533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1129/2242] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:04.581234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:04.602545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c5bb2-ca0a-4b66-8fd2-1e99f0cf6d53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:04.606583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c5bb2-ca0a-4b66-8fd2-1e99f0cf6d53 service swift] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:04.630576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dcb9c-a3e8-47bc-b6bb-4e1e7d6fcbfa admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:04.647647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1114/2243] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:05:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 08:05:04.705876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cca5d1-5053-4f80-826a-0b68dc4c0090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:04.714213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cca5d1-5053-4f80-826a-0b68dc4c0090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:04.725200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cca5d1-5053-4f80-826a-0b68dc4c0090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:04.725616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cca5d1-5053-4f80-826a-0b68dc4c0090 admin admin] PATH_INFO: `/v3/projects/ba57ec37209b4d9a919dfdfb1d2966f3/users/e7a6ab1f85224887a07234249d36ae03/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:04.801906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cca5d1-5053-4f80-826a-0b68dc4c0090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba57ec37209b4d9a919dfdfb1d2966f3, user_id=e7a6ab1f85224887a07234249d36ae03, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:04.804585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cca5d1-5053-4f80-826a-0b68dc4c0090 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:04.851088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1130/2244] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:04 2022] PUT /identity/v3/projects/ba57ec37209b4d9a919dfdfb1d2966f3/users/e7a6ab1f85224887a07234249d36ae03/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:04.864948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744ecb1-16cd-4d3c-abd7-6cedd135b8bf None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:04.865726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744ecb1-16cd-4d3c-abd7-6cedd135b8bf None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:04.866363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744ecb1-16cd-4d3c-abd7-6cedd135b8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:05.124103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-4744ecb1-16cd-4d3c-abd7-6cedd135b8bf None None] MFA Rules not processed for user `e7a6ab1f85224887a07234249d36ae03`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:05.132897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4744ecb1-16cd-4d3c-abd7-6cedd135b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:05.210104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:05.210411 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:05.210703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:05.319579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] MFA Rules not processed for user `8d82b72589ca4b72bac37042aad736cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:05.324763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:05.574570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1115/2245] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 712 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:05.801641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1131/2246] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:05 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 593 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:10.890698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab2e67-6414-45fa-ac75-0c33c09a2f56 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:10.898292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab2e67-6414-45fa-ac75-0c33c09a2f56 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:10.898997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab2e67-6414-45fa-ac75-0c33c09a2f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:11.023317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-50ab2e67-6414-45fa-ac75-0c33c09a2f56 None None] MFA Rules not processed for user `de6636a0522642799e43a8f02ee602ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:11.024646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-50ab2e67-6414-45fa-ac75-0c33c09a2f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:11.224400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:11.292198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1116/2247] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:11.296948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:11.298786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:11.465261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d546-cdae-4756-b283-9e2d081486e0 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:11.471680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:11.472848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:11.473262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:11.473578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:11.474441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:11.477265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:11.540361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1132/2248] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:05:11 2022] POST /identity/v3/projects => generated 371 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:11.544772 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab2e67-6414-45fa-ac75-0c33c09a2f56 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:11.569277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab2e67-6414-45fa-ac75-0c33c09a2f56 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:11.570629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:11.577235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:11.622066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:11.748800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:11.750113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:11.750560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:11.751033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:11.751977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:11.757104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:11.779614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368d43b-5f09-484d-a0c8-0e701eef0249 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:11.799157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:11.800076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:11.800716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:11.801094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] PATH_INFO: `/v3/users/d39918f5dad14060a8c659c3b3fd40dd` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:11.802650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39918f5dad14060a8c659c3b3fd40dd)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:11.804748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:11.809117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1117/2249] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:05:11 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:11.811177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:11.863856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096a274-f7e2-4894-abf3-c52aa8ef66ff admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:11.886431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:11.893446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:11.894505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:11.894505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:11.894858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:11.899726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:11.921196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39918f5dad14060a8c659c3b3fd40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:11.952997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39918f5dad14060a8c659c3b3fd40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:11.953319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1118/2250] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:11.953959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39918f5dad14060a8c659c3b3fd40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:11.956467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:11.963317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:11.984994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39918f5dad14060a8c659c3b3fd40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:11.986005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39918f5dad14060a8c659c3b3fd40dd'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:12.005034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25aab3-dd67-45f1-ba7f-f279e0178b2b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:12.005977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1133/2251] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:05:11 2022] DELETE /identity/v3/users/d39918f5dad14060a8c659c3b3fd40dd => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:12.236926 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:12.238785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:12.239248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:12.241459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] PATH_INFO: `/v3/projects/ce40207353eb449c99cb7abc1ddc878f/users/c1d7d84cf594405aa0e7f5307fa60657/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:12.400031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce40207353eb449c99cb7abc1ddc878f, user_id=c1d7d84cf594405aa0e7f5307fa60657, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:12.415391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:12.451289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1119/2252] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:11 2022] PUT /identity/v3/projects/ce40207353eb449c99cb7abc1ddc878f/users/c1d7d84cf594405aa0e7f5307fa60657/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:12.453028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:12.467474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf915d-f822-4aee-87be-70224e191012 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:12.576723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:12.577480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:12.577771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:12.578047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:12.578819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:12.580523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:12.607173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1134/2253] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:12.612312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:12.634141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa546-db27-49ee-8251-770f6fd0c474 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:12.649770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:12.651139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:12.651700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:12.652142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] PATH_INFO: `/v3/projects/ce40207353eb449c99cb7abc1ddc878f/users/c1d7d84cf594405aa0e7f5307fa60657/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:12.655006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce40207353eb449c99cb7abc1ddc878f, user_id=c1d7d84cf594405aa0e7f5307fa60657, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:12.664786 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:12.679449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:12.681344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:12.689491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1120/2254] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:12 2022] PUT /identity/v3/projects/ce40207353eb449c99cb7abc1ddc878f/users/c1d7d84cf594405aa0e7f5307fa60657/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:12.729644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6635e89-ab9b-48e7-b019-bd448f9e3ff4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:12.814683 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:12.973562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:12.975410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:12.975899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:12.976606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] PATH_INFO: `/v3/projects/7ff2f23d907640329a9468e514d8978a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:12.978241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff2f23d907640329a9468e514d8978a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:12.980760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:13.098803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff2f23d907640329a9468e514d8978a', 'request_id': 'req-5be35d03-09da-4469-944a-9ae0b08ab7f2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.105889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff2f23d907640329a9468e514d8978a', 'request_id': 'req-5be35d03-09da-4469-944a-9ae0b08ab7f2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.142357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff2f23d907640329a9468e514d8978a', 'request_id': 'req-5be35d03-09da-4469-944a-9ae0b08ab7f2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.163933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] The token cache is being invalidate because project 7ff2f23d907640329a9468e514d8978a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:05:13.164239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.169628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.174343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1135/2255] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:05:12 2022] DELETE /identity/v3/projects/7ff2f23d907640329a9468e514d8978a => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:13.193624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:13.194924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:13.237528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c164-c1fe-479a-82a5-97e5015c818a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:13.389383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:13.392870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:13.442272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:13.528926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35d03-09da-4469-944a-9ae0b08ab7f2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:13.534618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566471d-2c90-4122-a765-31e720d59bb4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:13.535626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566471d-2c90-4122-a765-31e720d59bb4 service glance] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:13.536469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566471d-2c90-4122-a765-31e720d59bb4 service glance] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:13.536936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566471d-2c90-4122-a765-31e720d59bb4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:13.539640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7566471d-2c90-4122-a765-31e720d59bb4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:13.600518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:13.602017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:13.602571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:13.603242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] PATH_INFO: `/v3/users/8d82b72589ca4b72bac37042aad736cc` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:13.605542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d82b72589ca4b72bac37042aad736cc)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:13.608737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:13.746503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d82b72589ca4b72bac37042aad736cc'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.747684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d82b72589ca4b72bac37042aad736cc'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.761258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566471d-2c90-4122-a765-31e720d59bb4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:13.768080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566471d-2c90-4122-a765-31e720d59bb4 service glance] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:13.782767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d82b72589ca4b72bac37042aad736cc'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.786984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d82b72589ca4b72bac37042aad736cc'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.827731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1136/2256] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:05:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:05:13.832897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdcc50-ad8a-4340-8602-bd8bb0e0d36a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:13.833433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdcc50-ad8a-4340-8602-bd8bb0e0d36a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:13.833723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdcc50-ad8a-4340-8602-bd8bb0e0d36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:13.857571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d82b72589ca4b72bac37042aad736cc'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:13.916441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1121/2257] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:05:13 2022] DELETE /identity/v3/users/8d82b72589ca4b72bac37042aad736cc => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:05:13.916730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-60fdcc50-ad8a-4340-8602-bd8bb0e0d36a None None] MFA Rules not processed for user `1f4af4eedee24897be2e702ef51873b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:13.918097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-60fdcc50-ad8a-4340-8602-bd8bb0e0d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:13.933186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:14.152775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1137/2258] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:14.957497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:14.976131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:15.197958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab227c-7de7-4654-a7fc-6aa77a293513 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:15.212372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:15.215614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:15.222996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:15.225125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] PATH_INFO: `/v3/projects/e21e901091484c4cbe5a5bc5ba8034b7` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:15.231881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e21e901091484c4cbe5a5bc5ba8034b7)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:15.237207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:15.427106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e21e901091484c4cbe5a5bc5ba8034b7', 'request_id': 'req-941fd8c9-a634-433e-9d8a-f8614c0c51b2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:15.443799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:15.450102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e21e901091484c4cbe5a5bc5ba8034b7', 'request_id': 'req-941fd8c9-a634-433e-9d8a-f8614c0c51b2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:15.536713 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e21e901091484c4cbe5a5bc5ba8034b7', 'request_id': 'req-941fd8c9-a634-433e-9d8a-f8614c0c51b2'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:15.619726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] The token cache is being invalidate because project e21e901091484c4cbe5a5bc5ba8034b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:05:15.623284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:15.628587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-941fd8c9-a634-433e-9d8a-f8614c0c51b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:05:15.642164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1122/2259] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:05:14 2022] DELETE /identity/v3/projects/e21e901091484c4cbe5a5bc5ba8034b7 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:15.650588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6d756-75c9-45bd-8a12-e0093825bf27 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:15.651474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6d756-75c9-45bd-8a12-e0093825bf27 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:15.651734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6d756-75c9-45bd-8a12-e0093825bf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:15.743063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4ab6d756-75c9-45bd-8a12-e0093825bf27 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:15.748132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4ab6d756-75c9-45bd-8a12-e0093825bf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:16.103952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1138/2260] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:16.113979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed796-2ebd-4145-aa1f-3f1b2ba35308 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:16.116586 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed796-2ebd-4145-aa1f-3f1b2ba35308 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:16.117189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed796-2ebd-4145-aa1f-3f1b2ba35308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:16.306077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8f4ed796-2ebd-4145-aa1f-3f1b2ba35308 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:16.312799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8f4ed796-2ebd-4145-aa1f-3f1b2ba35308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:16.378912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1123/2261] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:16.381286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6d756-75c9-45bd-8a12-e0093825bf27 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:16.425194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6d756-75c9-45bd-8a12-e0093825bf27 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:16.451599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:16.453010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:16.456254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:16.458094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:16.459070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:05:16.460582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:16.464761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:16.565582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1139/2262] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:05:16.567537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ed796-2ebd-4145-aa1f-3f1b2ba35308 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:16.621885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ed796-2ebd-4145-aa1f-3f1b2ba35308 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:16.663412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:16.664661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:16.665194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:16.665650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:16.668499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:16.680959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:16.846828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1124/2263] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:05:16 2022] POST /identity/v3/projects => generated 377 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:16.849279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:16.893545 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead1c8d-455c-46ad-b7e9-1b84af7639c4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:17.251235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:17.252406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:17.253708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:17.255069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:17.256330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:17.259302 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:17.337390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:17.348797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1140/2264] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:05:16 2022] POST /identity/v3/users => generated 289 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:17.462798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33677e-ec64-41ad-b042-b269aa29e979 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:17.502632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:17.505458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:17.505987 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:17.506504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:17.508025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:17.536474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:17.606835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:17.610845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1125/2265] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:17.635739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1df4e-cac8-42fe-b1c4-2d281be1270c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:17.674195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeda48-6484-44b5-8e0a-22e55e871818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:17.684766 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeda48-6484-44b5-8e0a-22e55e871818 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:17.685473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeda48-6484-44b5-8e0a-22e55e871818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:17.689526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeda48-6484-44b5-8e0a-22e55e871818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:17.697382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baeda48-6484-44b5-8e0a-22e55e871818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:17.709201 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baeda48-6484-44b5-8e0a-22e55e871818 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:17.817802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:17.818603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1141/2266] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:17.893439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3df3cf-4d7c-4bab-b52e-1a2d37bdc7e2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:17.915790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:17.917348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:17.917858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:17.918364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] PATH_INFO: `/v3/projects/60b844078680493c8efb657501f63946/users/06c1e590f8454eda90025881ee876fd5/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:18.020794 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b844078680493c8efb657501f63946, user_id=06c1e590f8454eda90025881ee876fd5, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:18.031045 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:18.077011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1126/2267] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:17 2022] PUT /identity/v3/projects/60b844078680493c8efb657501f63946/users/06c1e590f8454eda90025881ee876fd5/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:18.093910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b51f0-f36e-4f40-899b-834efbecf305 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:18.094638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b51f0-f36e-4f40-899b-834efbecf305 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:18.094638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b51f0-f36e-4f40-899b-834efbecf305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:18.260945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b53b51f0-f36e-4f40-899b-834efbecf305 None None] MFA Rules not processed for user `06c1e590f8454eda90025881ee876fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:18.278198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b53b51f0-f36e-4f40-899b-834efbecf305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:18.357973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:18.655213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1142/2268] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 562 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:05:18.661033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:18.829953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b88fc-1db4-4ab7-9209-9dcbf0fda890 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:18.854776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:18.862356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:18.863539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:18.864116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:18.876324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:18.886017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:19.031434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1127/2269] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:05:18 2022] POST /identity/v3/projects => generated 375 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:05:19.036867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b51f0-f36e-4f40-899b-834efbecf305 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:19.056885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b51f0-f36e-4f40-899b-834efbecf305 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:19.333326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:19.356841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:19.357377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:19.357613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:19.358974 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:19.371292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:19.462410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1143/2270] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:05:19 2022] POST /identity/v3/users => generated 294 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:05:19.465625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:19.509251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e2c8-044c-42ca-9fee-1af9d8c8a39a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:19.544238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:19.552016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:19.555461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:19.556121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:19.557723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:19.569354 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:19.594319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1128/2271] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:19.600547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:19.620296 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c727b4d-dd22-4bc4-87f0-00ed90af2d1b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:19.637970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:19.639864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:19.639864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:19.640124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] PATH_INFO: `/v3/projects/8fd18ea71ccd4365ba3354f041e002b2/users/d04ba05f4e4044029ca0595101b8dc0f/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:19.682244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd18ea71ccd4365ba3354f041e002b2, user_id=d04ba05f4e4044029ca0595101b8dc0f, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:19.686244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:19.730843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1144/2272] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:19 2022] PUT /identity/v3/projects/8fd18ea71ccd4365ba3354f041e002b2/users/d04ba05f4e4044029ca0595101b8dc0f/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:05:19.733464 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:19.795819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1c9e-e97a-4b75-9db8-cac26c5e659b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:20.119133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:20.182783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4679293-4cc9-42f7-b0ea-ee24b3e83305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:20.186189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4679293-4cc9-42f7-b0ea-ee24b3e83305 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:20.186654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4679293-4cc9-42f7-b0ea-ee24b3e83305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:20.187468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4679293-4cc9-42f7-b0ea-ee24b3e83305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:20.193445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4679293-4cc9-42f7-b0ea-ee24b3e83305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:20.203442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4679293-4cc9-42f7-b0ea-ee24b3e83305 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:20.269992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1129/2273] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:05:20.271949 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:20.289980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921c6e8-d4f4-4bad-93c9-bdd227bace8d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:20.331911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:20.334015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:20.334812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:20.335019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] PATH_INFO: `/v3/projects/8fd18ea71ccd4365ba3354f041e002b2/users/d04ba05f4e4044029ca0595101b8dc0f/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:20.342719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd18ea71ccd4365ba3354f041e002b2, user_id=d04ba05f4e4044029ca0595101b8dc0f, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:20.349427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:20.396688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1145/2274] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:05:20 2022] PUT /identity/v3/projects/8fd18ea71ccd4365ba3354f041e002b2/users/d04ba05f4e4044029ca0595101b8dc0f/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:05:20.409828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdbf61-f50c-4be6-af0a-9f0f2880a610 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:20.409828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdbf61-f50c-4be6-af0a-9f0f2880a610 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:20.410190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdbf61-f50c-4be6-af0a-9f0f2880a610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:20.608763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ccfdbf61-f50c-4be6-af0a-9f0f2880a610 None None] MFA Rules not processed for user `d04ba05f4e4044029ca0595101b8dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:20.619445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ccfdbf61-f50c-4be6-af0a-9f0f2880a610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:20.982104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1130/2275] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 576 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:20.992057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:20.995209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:21.212294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb21ced-bee3-42a2-9226-894f83423216 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:21.225460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:21.226173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:21.226459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:21.226734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:21.400449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:21.671183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:21.673477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:21.705439 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1146/2276] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:05:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:05:29.474084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf7392-b80e-431f-9fcd-9fcf7d0516b9 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:29.475043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf7392-b80e-431f-9fcd-9fcf7d0516b9 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:29.475901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf7392-b80e-431f-9fcd-9fcf7d0516b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:29.626934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6adf7392-b80e-431f-9fcd-9fcf7d0516b9 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:29.630035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6adf7392-b80e-431f-9fcd-9fcf7d0516b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:29.670075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:29.699400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce04a9-f5ab-4987-87ab-d999f8c2c4e3 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:29.729989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:29.730689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:29.732266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:29.732567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:29.781222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:29.783137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:29.845178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1147/2277] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:05:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:05:29.894976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1131/2278] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:05:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:29.928857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:29.930758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:30.348040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dbbd5-3ff4-4d17-8158-9af60d46f51a service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:30.403969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:30.405619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:30.408379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:30.410060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:30.477745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:30.522605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:30.577853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1148/2279] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:05:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:05:30.619055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49032e2b-a3d8-4734-b6e3-95aaf66c59ec None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:30.619917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49032e2b-a3d8-4734-b6e3-95aaf66c59ec None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:30.619917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49032e2b-a3d8-4734-b6e3-95aaf66c59ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:30.764631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-49032e2b-a3d8-4734-b6e3-95aaf66c59ec None None] MFA Rules not processed for user `c1d7d84cf594405aa0e7f5307fa60657`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:30.774637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-49032e2b-a3d8-4734-b6e3-95aaf66c59ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:31.080058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:31.154701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1132/2280] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:30 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 537 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:31.844838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:31.846935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:32.216731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bae43f-6211-4b52-b895-77db174e449e service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:32.243044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967d8dc-ad9a-4aa8-b8d1-41da6880cb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:32.243692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967d8dc-ad9a-4aa8-b8d1-41da6880cb20 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:32.243986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967d8dc-ad9a-4aa8-b8d1-41da6880cb20 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:32.244265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967d8dc-ad9a-4aa8-b8d1-41da6880cb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:32.248867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6967d8dc-ad9a-4aa8-b8d1-41da6880cb20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:32.278412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:32.631558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967d8dc-ad9a-4aa8-b8d1-41da6880cb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:32.634198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967d8dc-ad9a-4aa8-b8d1-41da6880cb20 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:32.672952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1149/2281] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:05:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 828 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:05:34.869897 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ebc90-4b94-4789-9dad-11bd544d1615 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:34.871514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ebc90-4b94-4789-9dad-11bd544d1615 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:34.872077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ebc90-4b94-4789-9dad-11bd544d1615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:34.940088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-323ebc90-4b94-4789-9dad-11bd544d1615 None None] MFA Rules not processed for user `6663857e60ca4440a9a620007be4c2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:34.941714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-323ebc90-4b94-4789-9dad-11bd544d1615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:35.195147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1133/2282] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:37.530081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa454d-e033-4d11-93e8-c575fbe02115 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:37.530961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa454d-e033-4d11-93e8-c575fbe02115 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:37.530961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa454d-e033-4d11-93e8-c575fbe02115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:37.571456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f2fa454d-e033-4d11-93e8-c575fbe02115 None None] MFA Rules not processed for user `de6636a0522642799e43a8f02ee602ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:37.572818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f2fa454d-e033-4d11-93e8-c575fbe02115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:37.606025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1150/2283] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:05:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:05:45.177307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=34 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:47.564096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=25 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:48.281771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:48.282344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:48.282793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:48.320175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:05:48.321741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:05:48.332685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1134/2284] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:05:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:05:48.350486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa454d-e033-4d11-93e8-c575fbe02115 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:05:48.366470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa454d-e033-4d11-93e8-c575fbe02115 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:05:48.374158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:05:48.393630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:05:48.393955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:05:48.394371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:05:48.404697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:05:48.406406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:05:48.417200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1151/2285] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:05:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 08:05:58.314572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:05:58.414861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=6 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:05.685824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:05.689027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:05.872267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a3fbf-4427-4fba-8353-39d432d9bd3b None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:05.882526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e72a8-06c3-4c2e-ae7f-030de9b2c234 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:05.883502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e72a8-06c3-4c2e-ae7f-030de9b2c234 service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:05.883962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e72a8-06c3-4c2e-ae7f-030de9b2c234 service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:05.884404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e72a8-06c3-4c2e-ae7f-030de9b2c234 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:05.911692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c99e72a8-06c3-4c2e-ae7f-030de9b2c234 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:05.935689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e72a8-06c3-4c2e-ae7f-030de9b2c234 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:05.937508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e72a8-06c3-4c2e-ae7f-030de9b2c234 service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:05.964180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1135/2286] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:14.004405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:14.086703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf26702-6bc0-40e3-a530-3944726bd707 service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:14.091389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:14.092310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:14.092586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:14.092858 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] PATH_INFO: `/v3/users/06c1e590f8454eda90025881ee876fd5` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:14.094345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c1e590f8454eda90025881ee876fd5)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:14.096489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:14.155384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c1e590f8454eda90025881ee876fd5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:14.174584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c1e590f8454eda90025881ee876fd5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:14.176034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c1e590f8454eda90025881ee876fd5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:14.185367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c1e590f8454eda90025881ee876fd5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:14.186010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c1e590f8454eda90025881ee876fd5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:14.200280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1152/2287] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:14 2022] DELETE /identity/v3/users/06c1e590f8454eda90025881ee876fd5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:14.206185 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b56043-b72e-499e-a418-e56f2aece989 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:14.206706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b56043-b72e-499e-a418-e56f2aece989 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:14.206816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b56043-b72e-499e-a418-e56f2aece989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:14.227410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-79b56043-b72e-499e-a418-e56f2aece989 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:14.229098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-79b56043-b72e-499e-a418-e56f2aece989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:14.312704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1136/2288] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:14.322766 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:14.324250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:14.431198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b07c5b-d9b2-45d3-b6de-4f0c9de62744 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:14.435778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1df0e-45c1-4d34-b751-67871a4ebc36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:14.436497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1df0e-45c1-4d34-b751-67871a4ebc36 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:14.436782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1df0e-45c1-4d34-b751-67871a4ebc36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:14.437125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1df0e-45c1-4d34-b751-67871a4ebc36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:14.451177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1df0e-45c1-4d34-b751-67871a4ebc36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:14.452669 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1df0e-45c1-4d34-b751-67871a4ebc36 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:14.486922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1153/2289] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:06:14.879261 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4fbf2-d017-4399-9a0d-a00a0c9b8e2f None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:14.880654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4fbf2-d017-4399-9a0d-a00a0c9b8e2f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:14.881574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4fbf2-d017-4399-9a0d-a00a0c9b8e2f None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:14.883315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1137/2290] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:06:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:06:14.891528 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c443-96d6-45c3-be01-c7906ba593ca None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:14.891912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c443-96d6-45c3-be01-c7906ba593ca None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:14.892705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c443-96d6-45c3-be01-c7906ba593ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:14.899072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:14.978203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-a9c6c443-96d6-45c3-be01-c7906ba593ca None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:14.980933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a9c6c443-96d6-45c3-be01-c7906ba593ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:15.099328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1154/2291] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:06:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 29 08:06:15.124743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4fbf2-d017-4399-9a0d-a00a0c9b8e2f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:15.128938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2ba4fbf2-d017-4399-9a0d-a00a0c9b8e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:15.381836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4fbf2-d017-4399-9a0d-a00a0c9b8e2f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:15.386799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af4082-5e5d-44c4-8c76-e4c2a5174cdf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:15.387747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4082-5e5d-44c4-8c76-e4c2a5174cdf service glance] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.388102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4082-5e5d-44c4-8c76-e4c2a5174cdf service glance] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.388452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4082-5e5d-44c4-8c76-e4c2a5174cdf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.402840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:15.407558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af4082-5e5d-44c4-8c76-e4c2a5174cdf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:15.410369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af4082-5e5d-44c4-8c76-e4c2a5174cdf service glance] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:15.427770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1138/2292] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:15.555312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc124-2d48-4972-bf24-271119cae704 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.555948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc124-2d48-4972-bf24-271119cae704 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.556723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc124-2d48-4972-bf24-271119cae704 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.560205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1155/2293] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:06:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:06:15.569052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ccce7-f083-4ea4-b208-cf006ee9cdf7 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.569589 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ccce7-f083-4ea4-b208-cf006ee9cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.570235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ccce7-f083-4ea4-b208-cf006ee9cdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.600559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ef1ccce7-f083-4ea4-b208-cf006ee9cdf7 None None] MFA Rules not processed for user `45a558f9c7c84a05a90595fda7fccd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:15.602091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ef1ccce7-f083-4ea4-b208-cf006ee9cdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:15.609953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1139/2294] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:06:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:15.623646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc124-2d48-4972-bf24-271119cae704 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:15.643216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc124-2d48-4972-bf24-271119cae704 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:15.647639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60d163-708e-4868-bf57-76f3692c8b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:15.648410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d163-708e-4868-bf57-76f3692c8b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.648701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d163-708e-4868-bf57-76f3692c8b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.649014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d163-708e-4868-bf57-76f3692c8b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.659994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60d163-708e-4868-bf57-76f3692c8b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:15.661530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60d163-708e-4868-bf57-76f3692c8b7d service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:15.671627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1156/2295] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:15.811710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb56d1c-94dd-4072-98c8-098ff18eb9bc None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.812416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb56d1c-94dd-4072-98c8-098ff18eb9bc None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.815385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb56d1c-94dd-4072-98c8-098ff18eb9bc None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.815919 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1140/2296] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:06:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:06:15.825986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e4dcb-20a9-4a9b-82ad-6cd01f01936a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.825986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e4dcb-20a9-4a9b-82ad-6cd01f01936a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.825986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e4dcb-20a9-4a9b-82ad-6cd01f01936a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.852926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4e4e4dcb-20a9-4a9b-82ad-6cd01f01936a None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:15.866491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4e4e4dcb-20a9-4a9b-82ad-6cd01f01936a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:15.880853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1157/2297] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:06:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:15.897717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb56d1c-94dd-4072-98c8-098ff18eb9bc None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:15.899974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dbb56d1c-94dd-4072-98c8-098ff18eb9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:15.910390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e4dcb-20a9-4a9b-82ad-6cd01f01936a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:15.919155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb56d1c-94dd-4072-98c8-098ff18eb9bc None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:15.925979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:15.927136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.928034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.928652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] PATH_INFO: `/v3/projects/60b844078680493c8efb657501f63946` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.928838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e4dcb-20a9-4a9b-82ad-6cd01f01936a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:15.930442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b844078680493c8efb657501f63946)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:15.932809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:15.935455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:15.936779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:15.937653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:15.938167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:15.961544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:15.963061 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:15.984695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1158/2298] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:16.016640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b844078680493c8efb657501f63946', 'request_id': 'req-0ba26e27-4249-4c08-9210-e9c1c8c013a4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.027432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b844078680493c8efb657501f63946', 'request_id': 'req-0ba26e27-4249-4c08-9210-e9c1c8c013a4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.043371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b844078680493c8efb657501f63946', 'request_id': 'req-0ba26e27-4249-4c08-9210-e9c1c8c013a4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.073675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] The token cache is being invalidate because project 60b844078680493c8efb657501f63946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:16.073896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.075410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.078566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1141/2299] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:15 2022] DELETE /identity/v3/projects/60b844078680493c8efb657501f63946 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:16.084624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:16.098262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ea68-e543-48a2-a757-e0e3a8beb3b8 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:16.210240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:16.213290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:16.215731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:16.216680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] PATH_INFO: `/v3/users/d04ba05f4e4044029ca0595101b8dc0f` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:16.219479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04ba05f4e4044029ca0595101b8dc0f)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:16.224258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:16.348625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04ba05f4e4044029ca0595101b8dc0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.352079 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:16.376207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04ba05f4e4044029ca0595101b8dc0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.380226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04ba05f4e4044029ca0595101b8dc0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.392447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04ba05f4e4044029ca0595101b8dc0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.393681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-31ecc4c1-9b09-4864-8d8e-7ba43136d4b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04ba05f4e4044029ca0595101b8dc0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:16.412180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1159/2300] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:16 2022] DELETE /identity/v3/users/d04ba05f4e4044029ca0595101b8dc0f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:17.171411 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:17.172845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:17.190322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26e27-4249-4c08-9210-e9c1c8c013a4 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:17.307659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:17.309326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:17.309873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:17.310395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] PATH_INFO: `/v3/projects/8fd18ea71ccd4365ba3354f041e002b2` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:17.312550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd18ea71ccd4365ba3354f041e002b2)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:17.316707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:17.387476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd18ea71ccd4365ba3354f041e002b2', 'request_id': 'req-fde23114-f2b5-41a1-b396-6830bdc32090'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:17.397157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd18ea71ccd4365ba3354f041e002b2', 'request_id': 'req-fde23114-f2b5-41a1-b396-6830bdc32090'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:17.404726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd18ea71ccd4365ba3354f041e002b2', 'request_id': 'req-fde23114-f2b5-41a1-b396-6830bdc32090'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:17.416735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:17.421578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] The token cache is being invalidate because project 8fd18ea71ccd4365ba3354f041e002b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:17.422211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:17.423259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fde23114-f2b5-41a1-b396-6830bdc32090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:17.427240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1142/2301] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:17 2022] DELETE /identity/v3/projects/8fd18ea71ccd4365ba3354f041e002b2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:17.432710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f007c-6c10-4863-aafb-1fb8b668c48f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:17.433355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f007c-6c10-4863-aafb-1fb8b668c48f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:17.433610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f007c-6c10-4863-aafb-1fb8b668c48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:17.453355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c88f007c-6c10-4863-aafb-1fb8b668c48f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:17.454657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c88f007c-6c10-4863-aafb-1fb8b668c48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:17.546864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1160/2302] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:17.556077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248668f-7e43-4681-8dc3-9d4619638d89 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:17.556209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248668f-7e43-4681-8dc3-9d4619638d89 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:17.556454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248668f-7e43-4681-8dc3-9d4619638d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:17.579118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6248668f-7e43-4681-8dc3-9d4619638d89 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:17.581293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6248668f-7e43-4681-8dc3-9d4619638d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:17.592103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1143/2303] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:06:17.597173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f007c-6c10-4863-aafb-1fb8b668c48f None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:17.607723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f007c-6c10-4863-aafb-1fb8b668c48f None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:17.613943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:17.614500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:17.614956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:17.615464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:17.616878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:06:17.617297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:17.618988 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:17.648068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1161/2304] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:06:17.653997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248668f-7e43-4681-8dc3-9d4619638d89 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:17.668478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248668f-7e43-4681-8dc3-9d4619638d89 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:17.676753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:17.677876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:17.678365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:17.679401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:17.680398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:17.683199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:17.732140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1144/2305] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:06:17 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:17.737696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:17.758770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6b310-4103-4ca4-a43a-ca178fd53edd admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:17.869402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:17.870422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:17.870913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:17.871203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:17.872265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:17.874539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:17.909090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1162/2306] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:06:17 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:06:17.917333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:17.934253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b4b58-d1ba-4aaf-8d2c-7d1210237756 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:17.942802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:17.943676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:17.944101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:17.944569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:17.945947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:17.948818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:17.971842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1145/2307] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:17.977353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:17.993861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e71bf-a91c-4744-b61c-c3508148a3b9 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.002432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bd9d1-7711-4155-b5c9-f3c9673994fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.004809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bd9d1-7711-4155-b5c9-f3c9673994fe admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.005368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bd9d1-7711-4155-b5c9-f3c9673994fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.005843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bd9d1-7711-4155-b5c9-f3c9673994fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.008136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bd9d1-7711-4155-b5c9-f3c9673994fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.011366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bd9d1-7711-4155-b5c9-f3c9673994fe admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.027656 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1163/2308] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:18.032140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:18.047533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c222c-8a41-45a4-8203-00dcace6ebdc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.054844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.056015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.056565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.057186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] PATH_INFO: `/v3/projects/8040efc94af146ff9f891773277564b7/users/433ef82d9eb54cd4ac8941ae9a506bb6/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.078092 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8040efc94af146ff9f891773277564b7, user_id=433ef82d9eb54cd4ac8941ae9a506bb6, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.081517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.098205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1146/2309] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:18 2022] PUT /identity/v3/projects/8040efc94af146ff9f891773277564b7/users/433ef82d9eb54cd4ac8941ae9a506bb6/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:18.116081 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494c17-570a-4959-9795-81747ced6df2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.116404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494c17-570a-4959-9795-81747ced6df2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.116684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494c17-570a-4959-9795-81747ced6df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.172266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e3494c17-570a-4959-9795-81747ced6df2 None None] MFA Rules not processed for user `433ef82d9eb54cd4ac8941ae9a506bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:18.174020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e3494c17-570a-4959-9795-81747ced6df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:18.214902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:18.275567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1164/2310] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:18.281031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:18.373426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb752-88fa-4793-a8a0-7defd245a6d9 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.381154 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.382079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.382363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.382642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.385443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.387423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.458967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1147/2311] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:06:18 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:06:18.463242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3494c17-570a-4959-9795-81747ced6df2 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:18.490805 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3494c17-570a-4959-9795-81747ced6df2 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.601784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.602877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.603930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.604327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.605174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.607331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.644438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1165/2312] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:06:18 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:18.649021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:18.662134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6298725-d69c-49b2-9b9b-5f90b6e86e0e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.666884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.667553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.667844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.668146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.668866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.670510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.683533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1148/2313] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:18.688386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:18.700502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f2f3a-55bc-478d-8810-c5116016552f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.707372 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.708670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.709129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.709516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] PATH_INFO: `/v3/projects/1d33dbc447a34f9686bce10474bafbc6/users/d2c4f79428654562aef78be5b3619864/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.734886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d33dbc447a34f9686bce10474bafbc6, user_id=d2c4f79428654562aef78be5b3619864, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.741985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.769293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1166/2314] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:18 2022] PUT /identity/v3/projects/1d33dbc447a34f9686bce10474bafbc6/users/d2c4f79428654562aef78be5b3619864/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:18.773106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:18.790338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a182a0-ea70-41e6-a706-aaada26845a7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.884130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63001203-f55a-4c0d-ae91-101b9b9f210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.884859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63001203-f55a-4c0d-ae91-101b9b9f210e admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.885177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63001203-f55a-4c0d-ae91-101b9b9f210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.885457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63001203-f55a-4c0d-ae91-101b9b9f210e admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.886258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63001203-f55a-4c0d-ae91-101b9b9f210e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.887945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63001203-f55a-4c0d-ae91-101b9b9f210e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.905612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1149/2315] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:06:18.911158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:18.927487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2e2d2-a035-49f4-84aa-3cec0a16c1df admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:18.935646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:18.937090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.937572 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.938033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] PATH_INFO: `/v3/projects/1d33dbc447a34f9686bce10474bafbc6/users/d2c4f79428654562aef78be5b3619864/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:18.942841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d33dbc447a34f9686bce10474bafbc6, user_id=d2c4f79428654562aef78be5b3619864, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:18.947924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:18.968596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1167/2316] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:18 2022] PUT /identity/v3/projects/1d33dbc447a34f9686bce10474bafbc6/users/d2c4f79428654562aef78be5b3619864/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:18.986615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:18.987168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:18.987779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:19.003824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:19.027032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] MFA Rules not processed for user `d2c4f79428654562aef78be5b3619864`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:19.028380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:19.132670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1150/2317] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:19.147202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:19.150784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:19.316036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc4fe-9ea9-44a5-827f-25ddc7fd6967 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:19.325761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:19.327449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:19.328294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:19.328919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:19.352706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:19.354560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:19.363623 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:19.387797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1168/2318] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:19.474153 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:19.476723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:19.549535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:19.698574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76708e-f600-4348-8b80-eaf9dc1122f3 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:19.715341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:19.716610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:19.716610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:19.716929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] PATH_INFO: `/v3/users/433ef82d9eb54cd4ac8941ae9a506bb6` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:19.718554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=433ef82d9eb54cd4ac8941ae9a506bb6)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:19.724148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:19.728766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73a26e-19fa-4a40-92aa-6abb746b9dc7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:19.738656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:19.740689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:19.741233 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:19.741747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] PATH_INFO: `/v3/users/e7a6ab1f85224887a07234249d36ae03` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:19.743692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a6ab1f85224887a07234249d36ae03)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:19.747209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:19.840677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433ef82d9eb54cd4ac8941ae9a506bb6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.866948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a6ab1f85224887a07234249d36ae03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.869152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a6ab1f85224887a07234249d36ae03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.870120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433ef82d9eb54cd4ac8941ae9a506bb6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.871601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433ef82d9eb54cd4ac8941ae9a506bb6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.884625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a6ab1f85224887a07234249d36ae03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.886177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a6ab1f85224887a07234249d36ae03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.895218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433ef82d9eb54cd4ac8941ae9a506bb6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.895969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9925d4ae-243b-4f71-8917-e4fad218ee66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433ef82d9eb54cd4ac8941ae9a506bb6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.921014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a6ead671-fc30-4859-8a41-89be0c588bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a6ab1f85224887a07234249d36ae03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:19.927438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1169/2319] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:19 2022] DELETE /identity/v3/users/433ef82d9eb54cd4ac8941ae9a506bb6 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:19.932543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:19.934118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:19.934118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:19.972941 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:19.977525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1151/2320] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:19 2022] DELETE /identity/v3/users/e7a6ab1f85224887a07234249d36ae03 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:19.979608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:19.979923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:19.980211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:19.990190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:20.018570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:20.023000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:20.037832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:20.194274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1170/2321] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:20.209620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1152/2322] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:20.211143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:20.214582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:20.229976 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:20.231511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:20.239348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:20.310739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f40d06-3b06-4028-9ec9-8cd59cc5f085 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:20.317221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:20.317905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:20.318240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:20.318467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:20.331965 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:20.333461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:20.345586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1171/2323] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:20.387003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c42291-da27-4cc3-bd84-5aaba7e41334 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:20.400120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:20.401628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:20.402830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:20.403303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:20.435197 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:20.439693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:20.469804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1153/2324] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:06:21.082068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:21.083859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:21.106625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7151f2-8b15-4683-b3ef-152002ca467d service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:21.115430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:21.116735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:21.117485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:21.118832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] PATH_INFO: `/v3/projects/8040efc94af146ff9f891773277564b7` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:21.131636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8040efc94af146ff9f891773277564b7)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:21.136292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:21.278505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8040efc94af146ff9f891773277564b7', 'request_id': 'req-06449438-c3ee-431e-bfbc-5a732435dcbf'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.289280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8040efc94af146ff9f891773277564b7', 'request_id': 'req-06449438-c3ee-431e-bfbc-5a732435dcbf'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.303024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8040efc94af146ff9f891773277564b7', 'request_id': 'req-06449438-c3ee-431e-bfbc-5a732435dcbf'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.330528 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] The token cache is being invalidate because project 8040efc94af146ff9f891773277564b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:21.330924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.331945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.334815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1172/2325] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:21 2022] DELETE /identity/v3/projects/8040efc94af146ff9f891773277564b7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:21.342853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:21.357036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c8b52-4ae7-4dd4-a78e-90999250c2a7 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:21.470844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:21.472131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:21.473266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:21.473675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] PATH_INFO: `/v3/users/d2c4f79428654562aef78be5b3619864` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:21.478550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c4f79428654562aef78be5b3619864)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:21.492536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:21.604503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:21.606161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:21.627068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449438-c3ee-431e-bfbc-5a732435dcbf admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:21.636109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c4f79428654562aef78be5b3619864'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.637161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c4f79428654562aef78be5b3619864'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.641726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:21.644278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:21.644597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:21.645854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] PATH_INFO: `/v3/projects/ba57ec37209b4d9a919dfdfb1d2966f3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:21.650526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba57ec37209b4d9a919dfdfb1d2966f3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:21.653346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c4f79428654562aef78be5b3619864'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.654182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c4f79428654562aef78be5b3619864'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.654931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:21.655690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:21.674370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-0e574be8-55d4-483c-ae99-78e465667079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c4f79428654562aef78be5b3619864'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.698155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1154/2326] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:21 2022] DELETE /identity/v3/users/d2c4f79428654562aef78be5b3619864 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:21.770471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba57ec37209b4d9a919dfdfb1d2966f3', 'request_id': 'req-3d161d08-7d14-47cf-9ec8-50a7e9675d60'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.788012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba57ec37209b4d9a919dfdfb1d2966f3', 'request_id': 'req-3d161d08-7d14-47cf-9ec8-50a7e9675d60'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.812094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba57ec37209b4d9a919dfdfb1d2966f3', 'request_id': 'req-3d161d08-7d14-47cf-9ec8-50a7e9675d60'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.838159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] The token cache is being invalidate because project ba57ec37209b4d9a919dfdfb1d2966f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:21.838653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.839493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3d161d08-7d14-47cf-9ec8-50a7e9675d60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:21.841593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1173/2327] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:21 2022] DELETE /identity/v3/projects/ba57ec37209b4d9a919dfdfb1d2966f3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:21.847492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5943280-5e42-40e0-80c3-2e130580bd9b None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:21.848273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5943280-5e42-40e0-80c3-2e130580bd9b None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:21.848747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5943280-5e42-40e0-80c3-2e130580bd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:21.877549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-d5943280-5e42-40e0-80c3-2e130580bd9b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:21.879282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d5943280-5e42-40e0-80c3-2e130580bd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:22.004504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1155/2328] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:06:22.015052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41703d4-37e8-46d5-83ce-e44225629fbb None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.015052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41703d4-37e8-46d5-83ce-e44225629fbb None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.015052 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41703d4-37e8-46d5-83ce-e44225629fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.046429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f41703d4-37e8-46d5-83ce-e44225629fbb None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:22.048670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f41703d4-37e8-46d5-83ce-e44225629fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:22.090593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5943280-5e42-40e0-80c3-2e130580bd9b None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.099902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1174/2329] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:22.104000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5943280-5e42-40e0-80c3-2e130580bd9b None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.109698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.110384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.110858 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.112351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.113218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:06:22.113538 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.115211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.130562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1156/2330] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:06:22.135380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41703d4-37e8-46d5-83ce-e44225629fbb None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.148846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41703d4-37e8-46d5-83ce-e44225629fbb None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.154966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.155724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.156003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.156461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.157465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.160821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.206599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:22.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1175/2331] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:06:22 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:06:22.227015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.241128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bbd35-c459-4cb8-b7d0-2436acba0993 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.356041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.357973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.357973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.357973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.358455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.360621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.402521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1157/2332] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:06:22 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:06:22.405576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.426235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008217fe-2079-45b2-b81f-5597dc9a8caf admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.432134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.433147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.433581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.434705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.436266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.449778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.482535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1176/2333] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:22.483850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.503092 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d599ac6-22d1-41af-9142-7105fde37853 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.512541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80697d6f-b73e-4488-900e-b0abc6519d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.514551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80697d6f-b73e-4488-900e-b0abc6519d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.515163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80697d6f-b73e-4488-900e-b0abc6519d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.515541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80697d6f-b73e-4488-900e-b0abc6519d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.516619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80697d6f-b73e-4488-900e-b0abc6519d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.520194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80697d6f-b73e-4488-900e-b0abc6519d6b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.536337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1158/2334] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:22.542308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.561838 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18643634-70e9-49f4-98e6-a942b8da62d9 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.570899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.572534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.573220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.573679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] PATH_INFO: `/v3/projects/2423b72f85274b5bbdeae539483eb699/users/1e7dcd8c7951473caf9f1bc61a58c473/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.608202 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2423b72f85274b5bbdeae539483eb699, user_id=1e7dcd8c7951473caf9f1bc61a58c473, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.618762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.641368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1177/2335] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:22 2022] PUT /identity/v3/projects/2423b72f85274b5bbdeae539483eb699/users/1e7dcd8c7951473caf9f1bc61a58c473/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:22.660595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.660964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.661322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.701573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] MFA Rules not processed for user `1e7dcd8c7951473caf9f1bc61a58c473`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:22.703023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:22.740780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:22.799501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1159/2336] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:22.812199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.816159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:22.857455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.860693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:22.916086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b2ca-1f19-48af-8149-9189145056a4 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.919867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.920537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.920820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.921131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.931292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.932754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.937798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0ba6a-acee-4fd0-ac50-9c8c5a0581eb None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.943754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1178/2337] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:22.944017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.945027 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.945408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.945806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] PATH_INFO: `/v3/projects/1d33dbc447a34f9686bce10474bafbc6` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.947322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d33dbc447a34f9686bce10474bafbc6)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:22.950020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:22.973491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:22.985552 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870b3c7-4d54-406a-8407-d990afa25136 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:22.995459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:22.996352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:22.996633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:22.996926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] PATH_INFO: `/v3/users/1e7dcd8c7951473caf9f1bc61a58c473` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:22.998165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e7dcd8c7951473caf9f1bc61a58c473)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.000117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.011346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d33dbc447a34f9686bce10474bafbc6', 'request_id': 'req-5430915e-6f36-48a2-abea-3727fb92f068'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.017784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d33dbc447a34f9686bce10474bafbc6', 'request_id': 'req-5430915e-6f36-48a2-abea-3727fb92f068'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.024610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d33dbc447a34f9686bce10474bafbc6', 'request_id': 'req-5430915e-6f36-48a2-abea-3727fb92f068'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.041654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] The token cache is being invalidate because project 1d33dbc447a34f9686bce10474bafbc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:23.041918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.045168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5430915e-6f36-48a2-abea-3727fb92f068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.048079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1160/2338] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:22 2022] DELETE /identity/v3/projects/1d33dbc447a34f9686bce10474bafbc6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:23.054309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e651a-7798-4167-8b4e-670bb140c86a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.054816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e651a-7798-4167-8b4e-670bb140c86a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.055243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e651a-7798-4167-8b4e-670bb140c86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.066077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e7dcd8c7951473caf9f1bc61a58c473'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.076324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-117e651a-7798-4167-8b4e-670bb140c86a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:23.077578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-117e651a-7798-4167-8b4e-670bb140c86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:23.088293 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e7dcd8c7951473caf9f1bc61a58c473'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.090866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e7dcd8c7951473caf9f1bc61a58c473'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.105990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e7dcd8c7951473caf9f1bc61a58c473'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.107046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5431e18d-ab3d-4a8b-a6c5-311723a14cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e7dcd8c7951473caf9f1bc61a58c473'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:23.122160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1179/2339] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:22 2022] DELETE /identity/v3/users/1e7dcd8c7951473caf9f1bc61a58c473 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:23.128035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.128328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.128844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.148702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:23.150114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:23.164920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:23.195573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1161/2340] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:23.197644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd212f0-1891-4941-9341-4d59353dd389 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.197644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd212f0-1891-4941-9341-4d59353dd389 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.197874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd212f0-1891-4941-9341-4d59353dd389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.213561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:23.236814 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-cfd212f0-1891-4941-9341-4d59353dd389 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:23.240787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-cfd212f0-1891-4941-9341-4d59353dd389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:23.289143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1180/2341] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:06:23.302920 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.304377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:23.326527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1162/2342] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:23.335218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd212f0-1891-4941-9341-4d59353dd389 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.356432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd212f0-1891-4941-9341-4d59353dd389 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.361402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.362090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.362373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.362646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.363390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:06:23.363697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.365356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.378371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b181491-1a0f-481c-b164-bfcd37192195 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.378872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1163/2343] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:06:23.383383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.385367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.386064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.386352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.386626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.396517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6152e-7098-4c4e-a1fe-bc75c6e40fcd admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.397229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.398698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.404625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.406767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.408149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.408677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.410078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.412290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1181/2344] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:06:23.413823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.449681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1164/2345] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:06:23 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:23.455707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.477639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8990-93a8-49a1-b9f3-fb5c5355988f service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.573083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.574771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.575721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.577681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.581292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.584880 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.613235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1182/2346] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:06:23 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:06:23.618017 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.643152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61990298-4354-4cf4-984f-58d2dfea5307 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.652857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.653989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.654506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.654962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.661312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.663369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.678529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1165/2347] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:23.682561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.694458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c3c9b-dc22-4d5b-a344-1a3a181b31ab admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.705231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.706438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.706906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.707417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.708821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.712638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.729475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1183/2348] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:06:23.734726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.752644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe4da-8e18-460d-be00-250d3f6c9b61 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.762009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2158d90-a59f-4906-8f22-64510798f4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.762009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2158d90-a59f-4906-8f22-64510798f4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.762009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2158d90-a59f-4906-8f22-64510798f4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.762009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2158d90-a59f-4906-8f22-64510798f4ab admin admin] PATH_INFO: `/v3/projects/473dbbe727c549a8939c3f6bc5333f02/users/960ac5f499d54a049b2c32fd5f2d2456/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.780357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2158d90-a59f-4906-8f22-64510798f4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=473dbbe727c549a8939c3f6bc5333f02, user_id=960ac5f499d54a049b2c32fd5f2d2456, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.784055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2158d90-a59f-4906-8f22-64510798f4ab admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.804009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1166/2349] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:23 2022] PUT /identity/v3/projects/473dbbe727c549a8939c3f6bc5333f02/users/960ac5f499d54a049b2c32fd5f2d2456/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:23.810208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.813389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:23.819755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5b786-cad0-41a2-9702-74f359f96bf5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.820112 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5b786-cad0-41a2-9702-74f359f96bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.820315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5b786-cad0-41a2-9702-74f359f96bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.832261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d976b5-e5b5-4feb-b64c-f6a3230d3d2c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:23.867835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3ab5b786-cad0-41a2-9702-74f359f96bf5 None None] MFA Rules not processed for user `960ac5f499d54a049b2c32fd5f2d2456`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:23.870068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3ab5b786-cad0-41a2-9702-74f359f96bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:23.930521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:23.931436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:23.931727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:23.932035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] PATH_INFO: `/v3/projects/2423b72f85274b5bbdeae539483eb699` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:23.933534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2423b72f85274b5bbdeae539483eb699)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:23.935561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:23.937528 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:23.971960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1167/2350] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:23.976454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5b786-cad0-41a2-9702-74f359f96bf5 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:23.989747 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5b786-cad0-41a2-9702-74f359f96bf5 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.010972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2423b72f85274b5bbdeae539483eb699', 'request_id': 'req-51b0b623-38a1-49bc-911a-f5e01bd78e1e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:24.018337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:24.020945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2423b72f85274b5bbdeae539483eb699', 'request_id': 'req-51b0b623-38a1-49bc-911a-f5e01bd78e1e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:24.032005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2423b72f85274b5bbdeae539483eb699', 'request_id': 'req-51b0b623-38a1-49bc-911a-f5e01bd78e1e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:24.057099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] The token cache is being invalidate because project 2423b72f85274b5bbdeae539483eb699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:24.057619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:24.058734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-51b0b623-38a1-49bc-911a-f5e01bd78e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:24.061605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1184/2351] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:23 2022] DELETE /identity/v3/projects/2423b72f85274b5bbdeae539483eb699 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:24.068619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4b505-e144-4169-bd68-fbd219a829f8 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.069097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4b505-e144-4169-bd68-fbd219a829f8 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.069608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4b505-e144-4169-bd68-fbd219a829f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.072400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.073118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.073430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.073689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.074396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.076368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.102387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-90f4b505-e144-4169-bd68-fbd219a829f8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:24.104658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-90f4b505-e144-4169-bd68-fbd219a829f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:24.106579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1168/2352] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:06:23 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:24.110734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.112872 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:24.196607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e471-9e68-4bee-9cc2-ee8a0e5c8ece admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.197580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1185/2353] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:24.206592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd248e-f002-4b84-9ef9-801eb1382a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.207326 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd248e-f002-4b84-9ef9-801eb1382a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.207613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd248e-f002-4b84-9ef9-801eb1382a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.207931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd248e-f002-4b84-9ef9-801eb1382a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.208686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd248e-f002-4b84-9ef9-801eb1382a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.210997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd248e-f002-4b84-9ef9-801eb1382a1e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.235814 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1169/2354] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:24.241085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4b505-e144-4169-bd68-fbd219a829f8 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.252764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4b505-e144-4169-bd68-fbd219a829f8 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.259959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.260947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.261433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.261826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.262693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.265348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.266688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79187102-cf42-459a-8d53-b08a3c7154fd None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.267287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79187102-cf42-459a-8d53-b08a3c7154fd None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.267672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79187102-cf42-459a-8d53-b08a3c7154fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.285862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1186/2355] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:24.290467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.300001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-79187102-cf42-459a-8d53-b08a3c7154fd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:24.301532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-79187102-cf42-459a-8d53-b08a3c7154fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:24.309272 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cfc7e-d4cf-45ae-a1b0-d41617833095 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.309863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1170/2356] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:24.314655 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79187102-cf42-459a-8d53-b08a3c7154fd None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.319541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.320266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.320547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.320820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.321598 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.323217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.325834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79187102-cf42-459a-8d53-b08a3c7154fd None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.331077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.331932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.332253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.332498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.333284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:06:24.333600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.335251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.341014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1187/2357] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:06:24.342080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.353639 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad6a1d-4dce-4418-9203-c445d9bb614c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.356248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1171/2358] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:06:24.361220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.363997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe44cce-d194-4732-a84c-9a2421635fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.365782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe44cce-d194-4732-a84c-9a2421635fae admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.366486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe44cce-d194-4732-a84c-9a2421635fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.367305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe44cce-d194-4732-a84c-9a2421635fae admin admin] PATH_INFO: `/v3/projects/0464ce291db34cc3b4eeff2e8311343a/users/e8e7942d8cf343019dd6edec5003b2e1/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.372364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116f927-7948-46b2-8f46-bae621d72ccd admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.379238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.379932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.380220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.380498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.381251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.384275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.395872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe44cce-d194-4732-a84c-9a2421635fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0464ce291db34cc3b4eeff2e8311343a, user_id=e8e7942d8cf343019dd6edec5003b2e1, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.402281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe44cce-d194-4732-a84c-9a2421635fae admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.425036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1188/2359] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:24 2022] PUT /identity/v3/projects/0464ce291db34cc3b4eeff2e8311343a/users/e8e7942d8cf343019dd6edec5003b2e1/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:24.432495 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1172/2360] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:24.436569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ba23-08d4-406e-bd91-6fb8ea94b052 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.436864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ba23-08d4-406e-bd91-6fb8ea94b052 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.437237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ba23-08d4-406e-bd91-6fb8ea94b052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.439567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.469073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e163ba23-08d4-406e-bd91-6fb8ea94b052 None None] MFA Rules not processed for user `e8e7942d8cf343019dd6edec5003b2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:24.470843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e163ba23-08d4-406e-bd91-6fb8ea94b052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:24.513878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:24.531484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:24.545876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa85f-6566-4e2e-8e51-9db8c093f228 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.552253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.553088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.553371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.553654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.554358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.556527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.565026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1189/2361] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:24.571166 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ba23-08d4-406e-bd91-6fb8ea94b052 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.581247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1173/2362] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:24.585224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.587390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ba23-08d4-406e-bd91-6fb8ea94b052 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.592266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.592938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.593647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.593937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.594641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.596845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560acbea-a0ed-4072-986d-bc0a586f97a5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.597162 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.603031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.603301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.603505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.603760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.604518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.606178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.621022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1174/2363] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:06:24.623001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.633161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1190/2364] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:24.634502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.638550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa22e2-cfb1-45ac-b357-91cfddc9d610 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.725220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.725823 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018197fe-6a85-4493-a366-0c92f2964010 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.725943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.726190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.726477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.727217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.729007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.730915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.731590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.732042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.732449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.733179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.735104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.741029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1175/2365] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:06:24.746950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.758830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1191/2366] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:06:24.762211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f10c-03a9-4ee2-b605-abacd197cd3d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.762870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.767850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71b074-f5bc-4856-9e23-35242294da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.768751 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71b074-f5bc-4856-9e23-35242294da6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.769070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71b074-f5bc-4856-9e23-35242294da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.769361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71b074-f5bc-4856-9e23-35242294da6e admin admin] PATH_INFO: `/v3/projects/774fad391358445c99bf5abd92281b58/users/6aa7adb1d11f42b4bf11ff230472773a/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.774754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb2cb5-f191-4a23-9fa0-57d6221d608e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.780298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.781418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.781704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.781979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.783494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.785223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.789151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71b074-f5bc-4856-9e23-35242294da6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=774fad391358445c99bf5abd92281b58, user_id=6aa7adb1d11f42b4bf11ff230472773a, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.791883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71b074-f5bc-4856-9e23-35242294da6e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.795891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1192/2367] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:06:24.800812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.808050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1176/2368] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:24 2022] PUT /identity/v3/projects/774fad391358445c99bf5abd92281b58/users/6aa7adb1d11f42b4bf11ff230472773a/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:24.815427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cde9cb-65de-48e7-beed-a230a74ac8eb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:24.819260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.819557 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.819839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.850051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] MFA Rules not processed for user `6aa7adb1d11f42b4bf11ff230472773a`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:24.851450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:24.883200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:24.884175 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:24.884467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:24.884775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] PATH_INFO: `/v3/projects/282c9a3258b8492fba731b2cfcb7e89a/users/20b5fc42bdc9464c920840405013d474/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:24.905473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282c9a3258b8492fba731b2cfcb7e89a, user_id=20b5fc42bdc9464c920840405013d474, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:24.908918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:24.924790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1193/2369] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:24 2022] PUT /identity/v3/projects/282c9a3258b8492fba731b2cfcb7e89a/users/20b5fc42bdc9464c920840405013d474/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:24.929146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.934687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1177/2370] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:06:24.942956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:24.946128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:24.947740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:24.983691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:24.996086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e056b20-099e-4476-b5d6-7e01e5ab6c07 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:25.000803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:25.001650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:25.001890 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:25.002170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:25.002909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:25.004521 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:25.007404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783cc97-499b-4d1a-a656-b1eed852ada8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:25.014650 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1194/2371] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:06:25.018745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:25.028247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c102eec-57a9-418d-94ef-8719f372d81c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:25.033463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445eb29a-49f8-42ca-b278-a81f4313fdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:25.034342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eb29a-49f8-42ca-b278-a81f4313fdbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:25.034630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eb29a-49f8-42ca-b278-a81f4313fdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:25.034909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eb29a-49f8-42ca-b278-a81f4313fdbf admin admin] PATH_INFO: `/v3/projects/282c9a3258b8492fba731b2cfcb7e89a/users/20b5fc42bdc9464c920840405013d474/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:25.036946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445eb29a-49f8-42ca-b278-a81f4313fdbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=282c9a3258b8492fba731b2cfcb7e89a, user_id=20b5fc42bdc9464c920840405013d474, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:25.039629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445eb29a-49f8-42ca-b278-a81f4313fdbf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:25.051626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1195/2372] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:06:25 2022] PUT /identity/v3/projects/282c9a3258b8492fba731b2cfcb7e89a/users/20b5fc42bdc9464c920840405013d474/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:25.062308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:25.063035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:25.063426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:25.063588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf4aa-580e-4384-8637-25dd935803be None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:25.063695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:25.063929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf4aa-580e-4384-8637-25dd935803be None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:25.064921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf4aa-580e-4384-8637-25dd935803be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:25.091664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-98bdf4aa-580e-4384-8637-25dd935803be None None] MFA Rules not processed for user `20b5fc42bdc9464c920840405013d474`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:25.092806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-98bdf4aa-580e-4384-8637-25dd935803be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:25.124101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:25.125585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:25.134891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1178/2373] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:06:25.163876 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1196/2374] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:25.175922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:25.177566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:25.188927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6e6d2-3533-4c5f-9407-09eaa297fe16 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:25.255985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2463c9a-60ea-49a6-a6e8-d521390c7853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:25.256853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2463c9a-60ea-49a6-a6e8-d521390c7853 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:25.257455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2463c9a-60ea-49a6-a6e8-d521390c7853 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:25.257773 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2463c9a-60ea-49a6-a6e8-d521390c7853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:25.269449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2463c9a-60ea-49a6-a6e8-d521390c7853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:25.271315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2463c9a-60ea-49a6-a6e8-d521390c7853 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:25.281834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1179/2375] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:26.037354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdf4aa-580e-4384-8637-25dd935803be None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:26.047555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdf4aa-580e-4384-8637-25dd935803be None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:26.052117 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:26.052796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:26.053133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:26.053388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:26.055445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:26.152333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:26.155031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:26.172380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1197/2376] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:26.546832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ff20-95dd-426c-b668-966875133217 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:26.547337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ff20-95dd-426c-b668-966875133217 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:26.547434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ff20-95dd-426c-b668-966875133217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:26.620813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6df5ff20-95dd-426c-b668-966875133217 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:26.622139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6df5ff20-95dd-426c-b668-966875133217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:26.642775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:26.711043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1180/2377] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:06:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:26.734709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:26.737416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:26.797198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:26.982836 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45ce0-1001-4b02-a095-b08df3d44afc service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:26.992499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:26.993308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:26.994309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:26.994600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:27.009864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:27.012059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:27.027903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1198/2378] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:06:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:06:32.678078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e118e-bd08-4ae7-8d32-76dd6b500cb0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:32.678782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e118e-bd08-4ae7-8d32-76dd6b500cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:32.678782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e118e-bd08-4ae7-8d32-76dd6b500cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:32.703057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-899e118e-bd08-4ae7-8d32-76dd6b500cb0 None None] MFA Rules not processed for user `c1feaba97f594ffda9dd2962bebdc082`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:32.705207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-899e118e-bd08-4ae7-8d32-76dd6b500cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:32.796187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1181/2379] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:06:34.169214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:34.208339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ec2c-5e70-443b-97b0-02d0ec160433 service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:34.237881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b7072-9b2e-4612-b2b0-cd757ae60cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:34.239320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b7072-9b2e-4612-b2b0-cd757ae60cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:34.240176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b7072-9b2e-4612-b2b0-cd757ae60cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:34.241305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b7072-9b2e-4612-b2b0-cd757ae60cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:34.414057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b7072-9b2e-4612-b2b0-cd757ae60cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:34.416415 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b7072-9b2e-4612-b2b0-cd757ae60cf3 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:34.434921 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1199/2380] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:37.716892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e118e-bd08-4ae7-8d32-76dd6b500cb0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:37.844247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e118e-bd08-4ae7-8d32-76dd6b500cb0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:37.864839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:37.865720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:37.866010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:37.866292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] PATH_INFO: `/v3/users/960ac5f499d54a049b2c32fd5f2d2456` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:37.870016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=960ac5f499d54a049b2c32fd5f2d2456)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:37.871479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:37.943262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960ac5f499d54a049b2c32fd5f2d2456'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:37.944314 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960ac5f499d54a049b2c32fd5f2d2456'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:37.963228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960ac5f499d54a049b2c32fd5f2d2456'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:37.964089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960ac5f499d54a049b2c32fd5f2d2456'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:37.974674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:37.990316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960ac5f499d54a049b2c32fd5f2d2456'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:38.037602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1182/2381] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:37 2022] DELETE /identity/v3/users/960ac5f499d54a049b2c32fd5f2d2456 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:38.039130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:38.039947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:38.040347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:38.087571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:38.090407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:38.265445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1200/2382] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:06:38.289335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:38.291952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:38.424435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae8cc1-f634-40a5-86e6-57b3342e6fdc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:38.432650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:38.433851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:38.434357 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:38.434845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:38.474459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:38.480954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:38.518631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1183/2383] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:39.050314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:39.055233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:39.086790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986a1b3-6d97-4a50-b71c-5f29c9f366bb None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:39.104797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:39.105915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:39.106299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:39.106581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] PATH_INFO: `/v3/projects/473dbbe727c549a8939c3f6bc5333f02` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:39.109090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=473dbbe727c549a8939c3f6bc5333f02)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:39.113164 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:39.149781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:39.179261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473dbbe727c549a8939c3f6bc5333f02', 'request_id': 'req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.187542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473dbbe727c549a8939c3f6bc5333f02', 'request_id': 'req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.196023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473dbbe727c549a8939c3f6bc5333f02', 'request_id': 'req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.208458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] The token cache is being invalidate because project 473dbbe727c549a8939c3f6bc5333f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:39.208745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.209750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.211711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1201/2384] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:39 2022] DELETE /identity/v3/projects/473dbbe727c549a8939c3f6bc5333f02 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:39.216171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:39.236064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93380ee-683e-45b9-a4c8-d61874691d40 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:39.323549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:39.324572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:39.325055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:39.325492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] PATH_INFO: `/v3/users/e8e7942d8cf343019dd6edec5003b2e1` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:39.327372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e7942d8cf343019dd6edec5003b2e1)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:39.330801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:39.397862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e7942d8cf343019dd6edec5003b2e1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.398568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e7942d8cf343019dd6edec5003b2e1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.407485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e7942d8cf343019dd6edec5003b2e1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.408356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e7942d8cf343019dd6edec5003b2e1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.422809 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e7942d8cf343019dd6edec5003b2e1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:39.440044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1184/2385] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:39 2022] DELETE /identity/v3/users/e8e7942d8cf343019dd6edec5003b2e1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:39.949143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:39.951672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:39.973557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88f7f9-dd5e-433f-a566-3f8120f11b0e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:40.056307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:40.057407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:40.057546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:40.057769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] PATH_INFO: `/v3/projects/0464ce291db34cc3b4eeff2e8311343a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:40.059104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0464ce291db34cc3b4eeff2e8311343a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:40.061323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:40.121758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0464ce291db34cc3b4eeff2e8311343a', 'request_id': 'req-0aca9455-67cc-47d8-8571-a8f324d9403a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.129069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0464ce291db34cc3b4eeff2e8311343a', 'request_id': 'req-0aca9455-67cc-47d8-8571-a8f324d9403a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.138894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0464ce291db34cc3b4eeff2e8311343a', 'request_id': 'req-0aca9455-67cc-47d8-8571-a8f324d9403a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.155354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] The token cache is being invalidate because project 0464ce291db34cc3b4eeff2e8311343a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:40.155642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.156321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.158707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1202/2386] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:39 2022] DELETE /identity/v3/projects/0464ce291db34cc3b4eeff2e8311343a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:40.163698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:40.166132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:40.184673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:40.194829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40e579-1531-4462-b956-3b3a76034854 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:40.308844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:40.310440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:40.310960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:40.311412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] PATH_INFO: `/v3/users/20b5fc42bdc9464c920840405013d474` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:40.314557 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b5fc42bdc9464c920840405013d474)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:40.318630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:40.404317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b5fc42bdc9464c920840405013d474'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.405534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b5fc42bdc9464c920840405013d474'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.421034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b5fc42bdc9464c920840405013d474'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.422445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b5fc42bdc9464c920840405013d474'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.447796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b5fc42bdc9464c920840405013d474'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:40.475628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1185/2387] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:40 2022] DELETE /identity/v3/users/20b5fc42bdc9464c920840405013d474 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:40.892336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:40.894562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:40.916300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca9455-67cc-47d8-8571-a8f324d9403a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:40.972485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:40.975079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:41.015031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:41.054060 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:41.055145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:41.055474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:41.055796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] PATH_INFO: `/v3/users/6aa7adb1d11f42b4bf11ff230472773a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:41.057755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa7adb1d11f42b4bf11ff230472773a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:41.060959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:41.132667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa7adb1d11f42b4bf11ff230472773a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.150720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038d83e-c32b-487f-a750-4120f87c3348 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:41.156225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa7adb1d11f42b4bf11ff230472773a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.158020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa7adb1d11f42b4bf11ff230472773a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.158885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:41.159807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:41.160099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:41.160375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] PATH_INFO: `/v3/projects/282c9a3258b8492fba731b2cfcb7e89a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:41.162064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282c9a3258b8492fba731b2cfcb7e89a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:41.164198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:41.173787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa7adb1d11f42b4bf11ff230472773a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.174734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5e45c232-64bf-4728-ad28-a617f46fb99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa7adb1d11f42b4bf11ff230472773a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.201079 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1203/2388] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:06:40 2022] DELETE /identity/v3/users/6aa7adb1d11f42b4bf11ff230472773a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:41.208005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:41.209006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:41.209770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:41.210291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:41.233534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282c9a3258b8492fba731b2cfcb7e89a', 'request_id': 'req-6704d7f0-25e1-417c-9073-664c30566432'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.241480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:41.241732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282c9a3258b8492fba731b2cfcb7e89a', 'request_id': 'req-6704d7f0-25e1-417c-9073-664c30566432'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.243619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:41.249209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282c9a3258b8492fba731b2cfcb7e89a', 'request_id': 'req-6704d7f0-25e1-417c-9073-664c30566432'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.282257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] The token cache is being invalidate because project 282c9a3258b8492fba731b2cfcb7e89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:41.283165 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.284172 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:41.286934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1186/2389] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:40 2022] DELETE /identity/v3/projects/282c9a3258b8492fba731b2cfcb7e89a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:06:41.345074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1204/2390] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:41.362478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:41.365435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:41.538564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704d7f0-25e1-417c-9073-664c30566432 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:41.547581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:41.549231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:41.549635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:41.550049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:41.564367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:41.566189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:41.581122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1187/2391] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:06:42.024485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:42.027033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:42.070324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbf5e9-d8f7-4ea5-a0e4-bdfd34c5ea47 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:42.075679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:42.077144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:42.077562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:42.077886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] PATH_INFO: `/v3/projects/774fad391358445c99bf5abd92281b58` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:42.080156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774fad391358445c99bf5abd92281b58)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:42.082266 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:42.137654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774fad391358445c99bf5abd92281b58', 'request_id': 'req-71f4ed54-b917-4f4d-9e39-2e5800845250'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:42.143455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:42.144796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774fad391358445c99bf5abd92281b58', 'request_id': 'req-71f4ed54-b917-4f4d-9e39-2e5800845250'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:42.154322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774fad391358445c99bf5abd92281b58', 'request_id': 'req-71f4ed54-b917-4f4d-9e39-2e5800845250'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:42.168334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] The token cache is being invalidate because project 774fad391358445c99bf5abd92281b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:06:42.168747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:42.169563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:06:42.171334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1205/2392] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:06:42 2022] DELETE /identity/v3/projects/774fad391358445c99bf5abd92281b58 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:06:51.533929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:51.553693 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96889ee-4392-4f03-b615-2c4599f77c43 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:51.631715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b4d51-465a-4113-ab3e-082c2dc6e246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:51.632688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b4d51-465a-4113-ab3e-082c2dc6e246 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:51.633231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b4d51-465a-4113-ab3e-082c2dc6e246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:51.633520 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b4d51-465a-4113-ab3e-082c2dc6e246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:51.635730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9c1b4d51-465a-4113-ab3e-082c2dc6e246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:51.758678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b4d51-465a-4113-ab3e-082c2dc6e246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:51.761070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b4d51-465a-4113-ab3e-082c2dc6e246 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:51.778545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1188/2393] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:52.065671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:52.068696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:52.113237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4ed54-b917-4f4d-9e39-2e5800845250 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:52.123095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:52.125506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:52.127393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:52.128165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:52.132208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:52.296078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:52.301000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:52.342691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1206/2394] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:06:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:06:57.036351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c24e4-eece-4c8a-b615-ab7b3e8f3929 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:57.036843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c24e4-eece-4c8a-b615-ab7b3e8f3929 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:57.037031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c24e4-eece-4c8a-b615-ab7b3e8f3929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:57.054913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:06:57.065444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-540c24e4-eece-4c8a-b615-ab7b3e8f3929 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:06:57.069338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-540c24e4-eece-4c8a-b615-ab7b3e8f3929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:57.178991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1189/2395] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jun 29 08:06:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:06:57.199691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:06:57.202112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:06:57.308313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef8099-2df9-49ea-a6f2-766883d3c993 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:06:57.315223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:06:57.316243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:06:57.317534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:06:57.318209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:06:57.350462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:06:57.352290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:06:57.377642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1207/2396] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:06:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 08:07:07.167271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=22 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:07:07.370130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=67,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:07:41.941934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822208c3-ed80-4f82-a902-3da19fe61233 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:41.942721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822208c3-ed80-4f82-a902-3da19fe61233 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:41.942721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822208c3-ed80-4f82-a902-3da19fe61233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:41.961315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-822208c3-ed80-4f82-a902-3da19fe61233 None None] MFA Rules not processed for user `1f4af4eedee24897be2e702ef51873b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:07:41.962526 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-822208c3-ed80-4f82-a902-3da19fe61233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:42.020558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1190/2397] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:07:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:07:48.695207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:48.697051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:48.773212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aadf2a-4d72-4101-b597-b1c83213a5e6 service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:48.777813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:48.778685 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:48.778964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:48.779234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] PATH_INFO: `/v3/users/c1feaba97f594ffda9dd2962bebdc082` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:48.780436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1feaba97f594ffda9dd2962bebdc082)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:48.782389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:48.833071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1feaba97f594ffda9dd2962bebdc082'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:48.845047 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1feaba97f594ffda9dd2962bebdc082'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:48.846530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1feaba97f594ffda9dd2962bebdc082'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:48.854427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1feaba97f594ffda9dd2962bebdc082'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:48.855140 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e47e08e9-2088-4749-8c4e-0f4d5d13f92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1feaba97f594ffda9dd2962bebdc082'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:48.869637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1208/2398] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:07:48 2022] DELETE /identity/v3/users/c1feaba97f594ffda9dd2962bebdc082 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:07:49.064074 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822208c3-ed80-4f82-a902-3da19fe61233 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.076256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822208c3-ed80-4f82-a902-3da19fe61233 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.138884 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.139749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.140037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.140316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] PATH_INFO: `/v3/projects/7bdadf97d44048c3bec51c0ae58ee4a6` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.141546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bdadf97d44048c3bec51c0ae58ee4a6)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.144021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.190422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bdadf97d44048c3bec51c0ae58ee4a6', 'request_id': 'req-5df064de-d4a6-47c7-a21d-03debfeee7a6'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:49.195916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bdadf97d44048c3bec51c0ae58ee4a6', 'request_id': 'req-5df064de-d4a6-47c7-a21d-03debfeee7a6'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:49.202064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bdadf97d44048c3bec51c0ae58ee4a6', 'request_id': 'req-5df064de-d4a6-47c7-a21d-03debfeee7a6'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:49.215113 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] The token cache is being invalidate because project 7bdadf97d44048c3bec51c0ae58ee4a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:07:49.215401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:49.216039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-5df064de-d4a6-47c7-a21d-03debfeee7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:49.217608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1191/2399] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:07:49 2022] DELETE /identity/v3/projects/7bdadf97d44048c3bec51c0ae58ee4a6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:07:49.223054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93882-44f5-4da4-80a5-20f00d174c52 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.223343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93882-44f5-4da4-80a5-20f00d174c52 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.223621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93882-44f5-4da4-80a5-20f00d174c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.241435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3da93882-44f5-4da4-80a5-20f00d174c52 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:07:49.242643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3da93882-44f5-4da4-80a5-20f00d174c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:49.304924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1209/2400] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:07:49.368300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f2b6-efab-4e3b-a57c-b8534490eb26 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.368506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f2b6-efab-4e3b-a57c-b8534490eb26 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.368801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f2b6-efab-4e3b-a57c-b8534490eb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.388712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6817f2b6-efab-4e3b-a57c-b8534490eb26 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:07:49.389972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6817f2b6-efab-4e3b-a57c-b8534490eb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:49.399334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1192/2401] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:07:49.403658 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da93882-44f5-4da4-80a5-20f00d174c52 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.414338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da93882-44f5-4da4-80a5-20f00d174c52 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.419301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.420105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.420384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.420662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.421422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:07:49.421723 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.423463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.435811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1210/2402] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:07:49.440511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f2b6-efab-4e3b-a57c-b8534490eb26 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.450116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f2b6-efab-4e3b-a57c-b8534490eb26 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.455821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.456629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.456920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.457298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.458278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.460355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.494922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1193/2403] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:07:49 2022] POST /identity/v3/projects => generated 397 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:07:49.499631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.510294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b2b83-c6b0-4291-bacb-ee87908a52c1 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.513917 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:07:49.570220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.570888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.571177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.571451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.572139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.574168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.612252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1211/2404] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:07:49 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:07:49.613652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.629071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e6fd-9698-44e7-844b-8649928726a4 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.634848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.635528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.635811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.636086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.637012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.638732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.640273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:07:49.655236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1194/2405] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:07:49.657045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.668484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdd5e2-1510-407a-ad08-0fff111463f0 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.674027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6b21b-e32f-49eb-9113-5a371781acc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.674883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6b21b-e32f-49eb-9113-5a371781acc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.675221 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6b21b-e32f-49eb-9113-5a371781acc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.675715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6b21b-e32f-49eb-9113-5a371781acc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.676454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6b21b-e32f-49eb-9113-5a371781acc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.678103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6b21b-e32f-49eb-9113-5a371781acc7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.688520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1212/2406] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:07:49.693282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.703396 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eecbbe-cfa1-4b42-8135-99fb46e8aa43 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.708667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.709592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.709876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.710150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] PATH_INFO: `/v3/projects/52f07702d657486ca5d6c8ef750cb5cf/users/5bb06ddad79948f8abf1450f661a6d23/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.725824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f07702d657486ca5d6c8ef750cb5cf, user_id=5bb06ddad79948f8abf1450f661a6d23, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.728503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.743729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1195/2407] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:07:49 2022] PUT /identity/v3/projects/52f07702d657486ca5d6c8ef750cb5cf/users/5bb06ddad79948f8abf1450f661a6d23/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:07:49.754708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.755250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.755250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.784578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] MFA Rules not processed for user `5bb06ddad79948f8abf1450f661a6d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:07:49.785943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:49.854789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1213/2408] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:07:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:07:49.868122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:49.869813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:49.938014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78cba2-d61f-49a3-89be-14e588ed8fa2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:49.942023 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:49.942678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:49.942964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:49.943316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:49.952118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:49.953591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:49.963923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1196/2409] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:07:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:07:54.926298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:54.927859 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:54.991465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab87fc1-2f54-4301-b173-5161e8630c9a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:54.995642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13e065-cabd-4f73-9fc9-9e9e8eb973a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:54.996501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e065-cabd-4f73-9fc9-9e9e8eb973a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:54.997001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e065-cabd-4f73-9fc9-9e9e8eb973a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:54.997429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e065-cabd-4f73-9fc9-9e9e8eb973a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:54.999779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9a13e065-cabd-4f73-9fc9-9e9e8eb973a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:55.041893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:07:55.090311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13e065-cabd-4f73-9fc9-9e9e8eb973a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:55.091768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13e065-cabd-4f73-9fc9-9e9e8eb973a4 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:55.104738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1214/2410] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:07:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:07:58.122909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:58.213573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18fe90-fd81-4e4c-b2b4-3dd442e63df6 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:58.215952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:58.217024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:58.217408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:58.217790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] PATH_INFO: `/v3/users/5bb06ddad79948f8abf1450f661a6d23` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:58.219180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb06ddad79948f8abf1450f661a6d23)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:58.222524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:58.292470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb06ddad79948f8abf1450f661a6d23'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:58.293324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb06ddad79948f8abf1450f661a6d23'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:58.303296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb06ddad79948f8abf1450f661a6d23'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:58.304143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb06ddad79948f8abf1450f661a6d23'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:58.329077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb06ddad79948f8abf1450f661a6d23'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:58.333084 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:07:58.357067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1197/2411] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:07:58 2022] DELETE /identity/v3/users/5bb06ddad79948f8abf1450f661a6d23 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:07:58.359023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:58.360015 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:58.361057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:58.392681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:07:58.394170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:58.482947 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1215/2412] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:07:58.528937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:58.530859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:58.653155 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d90b2-3db9-4dcc-ae7a-9c55483b6435 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:58.666259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:58.667089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:58.667474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:58.667854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:58.680511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:58.682194 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:58.696270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1198/2413] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:07:58.750721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:58.753004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:58.912200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195469-4158-44d8-9f9d-7c59367eb11e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:58.919878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde2d89-3883-485b-8bb0-fa2b8fd52a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:58.920834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2d89-3883-485b-8bb0-fa2b8fd52a97 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:58.921326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2d89-3883-485b-8bb0-fa2b8fd52a97 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:58.921608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2d89-3883-485b-8bb0-fa2b8fd52a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:58.928657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-adde2d89-3883-485b-8bb0-fa2b8fd52a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:59.094573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:07:59.103750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde2d89-3883-485b-8bb0-fa2b8fd52a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:59.107485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde2d89-3883-485b-8bb0-fa2b8fd52a97 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:59.145224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1216/2414] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:07:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:07:59.488455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:07:59.492723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:07:59.513502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7daf33-e017-46e9-b77f-0e13b21a4a98 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:07:59.521534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:07:59.522588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:07:59.523022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:07:59.523529 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] PATH_INFO: `/v3/projects/52f07702d657486ca5d6c8ef750cb5cf` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:07:59.525598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f07702d657486ca5d6c8ef750cb5cf)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:07:59.529801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:07:59.609449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f07702d657486ca5d6c8ef750cb5cf', 'request_id': 'req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:59.619372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f07702d657486ca5d6c8ef750cb5cf', 'request_id': 'req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:59.631367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f07702d657486ca5d6c8ef750cb5cf', 'request_id': 'req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:59.657039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] The token cache is being invalidate because project 52f07702d657486ca5d6c8ef750cb5cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:07:59.657546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:59.658441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fa383d3d-9df9-4d8e-96b3-2aa9481d1314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:07:59.661031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1199/2415] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:07:59 2022] DELETE /identity/v3/projects/52f07702d657486ca5d6c8ef750cb5cf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:03.127062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:03.127599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:03.127599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:03.145952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] MFA Rules not processed for user `c70c44993ab240b1a5231df4c9d550c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:03.147328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.156679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:03.157730 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:03.157854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:03.176828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] MFA Rules not processed for user `c9065575ed314f6f8f3915bf596911e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:03.177842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.208824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1217/2416] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:08:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:08:03.240932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1200/2417] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:08:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:08:03.247835 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:03.249352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.316742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c80a63-0c64-4bce-9ab8-f51f843578ed None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:03.320428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:03.321125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:03.321412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:03.321692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:03.323526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.404349 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:03.405817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:03.421161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1218/2418] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:03.583050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:03.584716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.622901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:03.666824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb8d91-50e8-4d48-bfa4-acb61543b380 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:03.670633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:03.671406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:03.671761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:03.672134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:03.675968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.690489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:03.691971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:03.700954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1201/2419] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:03.747290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:03.748897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.764797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd129af-28b3-40ff-8fb3-2cfb549196e1 service glance] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:03.771184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:03.772328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:03.773288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:03.773413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:03.776541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:03.793551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:03.795010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:03.806121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1219/2420] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:04.033841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:04.035618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:04.057538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:04.059121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:04.061248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6004b6d-1e49-4474-93f7-c689cd77bf1b service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:04.069703 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:04.070595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:04.071679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:04.071679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:04.073727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:04.085079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:04.086742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:04.097441 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:04.098239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1202/2421] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:04.161271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:04.162782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:04.178506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac1473-bdcf-4af3-b5ae-9ac221194470 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:04.184024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:04.184821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:04.185212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:04.185564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:04.187729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:04.191195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94a45-6dbd-43cc-b0a6-c4f8d798f9fa service nova] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:04.196709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:04.197488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:04.197952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:04.198306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:04.202283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:04.204613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:04.213132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:04.213132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:04.221367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1220/2422] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:08:04.226504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1203/2423] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:08:14.215187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=20 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:14.218710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=27 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:17.828273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:17.846237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0e9d3-034d-4309-ab85-b2bb9425d827 service placement] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:17.852913 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:17.854104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:17.854605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:17.855110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:17.858500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:17.878523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:17.881565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:17.898733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1221/2424] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:25.891932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:25.893696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:25.970732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8bb96-6d08-4657-82de-a5ec19ef542b service placement] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:25.976274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e820a-46b2-4497-95e6-fb2f69ea1630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:25.976967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e820a-46b2-4497-95e6-fb2f69ea1630 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:25.977284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e820a-46b2-4497-95e6-fb2f69ea1630 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:25.977566 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e820a-46b2-4497-95e6-fb2f69ea1630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:25.979568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8a5e820a-46b2-4497-95e6-fb2f69ea1630 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:26.058455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e820a-46b2-4497-95e6-fb2f69ea1630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:26.060107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e820a-46b2-4497-95e6-fb2f69ea1630 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:26.071109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1204/2425] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:26.736450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:26.749578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b717d-9b18-4367-8261-bf611c2a8d20 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:26.755977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:26.756725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:26.757171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:26.757578 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:26.759519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:26.773020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:26.774622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:26.786596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1222/2426] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:27.858342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:27.858875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:27.860757 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:27.886548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:27.887966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:27.974229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1205/2427] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jun 29 08:08:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:08:28.028810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:28.030283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:28.104496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b1d83-f755-4876-b827-49b29490243e service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:28.110096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2035-e77f-421d-b902-a89561953deb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:28.110894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2035-e77f-421d-b902-a89561953deb service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:28.111188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2035-e77f-421d-b902-a89561953deb service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:28.111462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2035-e77f-421d-b902-a89561953deb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:28.120245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a2035-e77f-421d-b902-a89561953deb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:28.121871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a2035-e77f-421d-b902-a89561953deb service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:28.130684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1223/2428] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:08:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 08:08:37.317979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:37.319442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:37.396766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e895b-8daa-41d0-9782-0e119a6d9aa2 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:37.401389 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:37.402272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:37.402556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:37.402837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] PATH_INFO: `/v3/users/6663857e60ca4440a9a620007be4c2c7` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:37.404063 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6663857e60ca4440a9a620007be4c2c7)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:37.407333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:37.409137 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:37.469241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6663857e60ca4440a9a620007be4c2c7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:37.470028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6663857e60ca4440a9a620007be4c2c7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:37.479762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6663857e60ca4440a9a620007be4c2c7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:37.480603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6663857e60ca4440a9a620007be4c2c7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:37.503740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6663857e60ca4440a9a620007be4c2c7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:37.521853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1206/2429] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:08:37 2022] DELETE /identity/v3/users/6663857e60ca4440a9a620007be4c2c7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:37.808620 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2035-e77f-421d-b902-a89561953deb service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:37.822673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2035-e77f-421d-b902-a89561953deb service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:37.943477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:37.944368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:37.944651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:37.944929 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] PATH_INFO: `/v3/projects/a7afb1bdb06042f4b983631745f32e19` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:37.946160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7afb1bdb06042f4b983631745f32e19)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:37.948110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:38.006181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7afb1bdb06042f4b983631745f32e19', 'request_id': 'req-c28f038a-012c-4952-be65-fe16b1856bae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.011914 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7afb1bdb06042f4b983631745f32e19', 'request_id': 'req-c28f038a-012c-4952-be65-fe16b1856bae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.020904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7afb1bdb06042f4b983631745f32e19', 'request_id': 'req-c28f038a-012c-4952-be65-fe16b1856bae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.022205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:38.037590 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] The token cache is being invalidate because project a7afb1bdb06042f4b983631745f32e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:08:38.038055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.038879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.044028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1224/2430] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:08:37 2022] DELETE /identity/v3/projects/a7afb1bdb06042f4b983631745f32e19 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:38.048740 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:38.050252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:38.064375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72419c9a-3727-44aa-b823-309f1a180446 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:38.130770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:38.131675 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:38.131959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:38.132239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] PATH_INFO: `/v3/users/35b8945cc3014361af5fc2c7f92db8a5` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:38.133584 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b8945cc3014361af5fc2c7f92db8a5)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:38.135616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:38.205528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b8945cc3014361af5fc2c7f92db8a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.206293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b8945cc3014361af5fc2c7f92db8a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.214171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b8945cc3014361af5fc2c7f92db8a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.214939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b8945cc3014361af5fc2c7f92db8a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.229939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b8945cc3014361af5fc2c7f92db8a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.244039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:38.251313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1207/2431] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:08:38 2022] DELETE /identity/v3/users/35b8945cc3014361af5fc2c7f92db8a5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:38.560868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:38.562768 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:38.588452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f038a-012c-4952-be65-fe16b1856bae admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:38.666322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:38.667218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:38.667509 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:38.667791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] PATH_INFO: `/v3/projects/02814cee7f27425b848a6dfc38e5a714` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:38.669133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02814cee7f27425b848a6dfc38e5a714)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:38.671099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:38.728003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02814cee7f27425b848a6dfc38e5a714', 'request_id': 'req-4561afc3-dde8-40cc-b034-49c37922844a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.735734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02814cee7f27425b848a6dfc38e5a714', 'request_id': 'req-4561afc3-dde8-40cc-b034-49c37922844a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.745219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02814cee7f27425b848a6dfc38e5a714', 'request_id': 'req-4561afc3-dde8-40cc-b034-49c37922844a'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.759009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] The token cache is being invalidate because project 02814cee7f27425b848a6dfc38e5a714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:08:38.759357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.760106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:38.761870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1225/2432] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:08:38 2022] DELETE /identity/v3/projects/02814cee7f27425b848a6dfc38e5a714 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:46.124499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:46.124499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:46.189673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82afc98-b1d7-4806-8b1c-c63f908b94af admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:46.258031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:46.259064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:46.259631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:46.260148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:46.263071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:46.342191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:46.346221 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:46.364236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1208/2433] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:08:48.755611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:49.835323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:49.836814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:49.854158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561afc3-dde8-40cc-b034-49c37922844a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:49.864198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:49.865145 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:49.865430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:49.865703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] PATH_INFO: `/v3/users/1f4af4eedee24897be2e702ef51873b3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:49.866906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4af4eedee24897be2e702ef51873b3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:49.868852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:49.921216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4af4eedee24897be2e702ef51873b3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:49.933879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4af4eedee24897be2e702ef51873b3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:49.934631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4af4eedee24897be2e702ef51873b3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:49.942557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4af4eedee24897be2e702ef51873b3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:49.943177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a67983c6-6e5e-4a68-a7e2-25228ad74272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4af4eedee24897be2e702ef51873b3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:49.956473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1226/2434] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:08:49 2022] DELETE /identity/v3/users/1f4af4eedee24897be2e702ef51873b3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:50.151082 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.163437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befca07-c6d6-4d11-901a-403f630330e5 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.230958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.231840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.232129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.232413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] PATH_INFO: `/v3/projects/ab693125979f4bb7b7b3c94061610fe2` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.233665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab693125979f4bb7b7b3c94061610fe2)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.235998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:50.242816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:50.287147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab693125979f4bb7b7b3c94061610fe2', 'request_id': 'req-59c61f2a-af30-4902-886b-abafe0fd28c3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:50.292880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab693125979f4bb7b7b3c94061610fe2', 'request_id': 'req-59c61f2a-af30-4902-886b-abafe0fd28c3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:50.299383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab693125979f4bb7b7b3c94061610fe2', 'request_id': 'req-59c61f2a-af30-4902-886b-abafe0fd28c3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:50.312329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] The token cache is being invalidate because project ab693125979f4bb7b7b3c94061610fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:08:50.312622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:50.314033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-59c61f2a-af30-4902-886b-abafe0fd28c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:08:50.315632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1209/2435] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:08:50 2022] DELETE /identity/v3/projects/ab693125979f4bb7b7b3c94061610fe2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:50.320677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c4db3-31f3-4500-92aa-0c07c37dba04 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.320964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c4db3-31f3-4500-92aa-0c07c37dba04 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.321277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c4db3-31f3-4500-92aa-0c07c37dba04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.342518 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-436c4db3-31f3-4500-92aa-0c07c37dba04 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:50.343649 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-436c4db3-31f3-4500-92aa-0c07c37dba04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:50.407773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1227/2436] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:08:50.415579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067728-d104-4c01-b1a2-897384c95295 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.415961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067728-d104-4c01-b1a2-897384c95295 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.416277 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067728-d104-4c01-b1a2-897384c95295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.438434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-aa067728-d104-4c01-b1a2-897384c95295 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:50.439882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-aa067728-d104-4c01-b1a2-897384c95295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:50.448990 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1210/2437] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:08:50.454473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c4db3-31f3-4500-92aa-0c07c37dba04 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.464883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c4db3-31f3-4500-92aa-0c07c37dba04 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.471295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.471977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.472265 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.472542 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.473327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:08:50.473636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.475261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:50.495004 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1228/2438] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:08:50.499532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa067728-d104-4c01-b1a2-897384c95295 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.511026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa067728-d104-4c01-b1a2-897384c95295 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.516493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.517904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.518193 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.518475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.519199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.521157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:50.558057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1211/2439] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:08:50 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:08:50.559462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.571142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e16690-26c8-47f9-a64b-36512e55bc1e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.633534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.634132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.634418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.634704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.635409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.637371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:50.645260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:50.668853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1229/2440] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:08:50 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:08:50.671068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.683311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d9704-8009-4af1-9770-dd341173d6f7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.689504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.690202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.690492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.690974 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.691934 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.694059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:50.706438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1212/2441] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:08:50.710577 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.721994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a787385-8736-4936-9ef5-d5855b173179 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.728342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843208d-072a-43cc-9fa5-b420ac73328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.729257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843208d-072a-43cc-9fa5-b420ac73328f admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.729715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843208d-072a-43cc-9fa5-b420ac73328f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.730192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843208d-072a-43cc-9fa5-b420ac73328f admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.731297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843208d-072a-43cc-9fa5-b420ac73328f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.733214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843208d-072a-43cc-9fa5-b420ac73328f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:50.746897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1230/2442] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:08:50.752473 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.763083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1094-a050-43de-9c2c-5fde8b3b610f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.768099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.770425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.770425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.770425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] PATH_INFO: `/v3/projects/a4df15f6a6dd45dab12d0b724076a23e/users/4873db0f2fbf4a77a617f2102a5ccfae/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.784656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4df15f6a6dd45dab12d0b724076a23e, user_id=4873db0f2fbf4a77a617f2102a5ccfae, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.787353 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:50.802671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1213/2443] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:08:50 2022] PUT /identity/v3/projects/a4df15f6a6dd45dab12d0b724076a23e/users/4873db0f2fbf4a77a617f2102a5ccfae/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:08:50.807802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7774e-8926-4497-9979-a32150d15c27 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.808087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7774e-8926-4497-9979-a32150d15c27 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.808366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7774e-8926-4497-9979-a32150d15c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.826841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fde7774e-8926-4497-9979-a32150d15c27 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:50.828067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fde7774e-8926-4497-9979-a32150d15c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:50.893500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1231/2444] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:08:50.903251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:50.905787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:50.976602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad66f44-b3d5-4481-b153-0501ef2ab7ce admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:50.980294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:50.980951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:50.981266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:50.981543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:50.992099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:50.993575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:51.003784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1214/2445] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:56.372220 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1279e-8b63-4ab3-a914-b1afc4038f84 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:56.373072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1279e-8b63-4ab3-a914-b1afc4038f84 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:56.373072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1279e-8b63-4ab3-a914-b1afc4038f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:56.414640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8fc1279e-8b63-4ab3-a914-b1afc4038f84 None None] MFA Rules not processed for user `4873db0f2fbf4a77a617f2102a5ccfae`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:56.416176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8fc1279e-8b63-4ab3-a914-b1afc4038f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:56.575224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1232/2446] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:08:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:08:56.585555 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:08:56.587468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:56.671237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:08:56.734234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481aa9b-1c88-42be-828d-76d27bd4a659 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:08:56.745775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:08:56.748105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:56.748105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:56.748105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:56.757311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:08:56.758944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:08:56.770627 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1215/2447] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:08:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:08:58.300186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71437d52-283d-42a5-b9ed-13b440bac4a4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:08:58.301046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71437d52-283d-42a5-b9ed-13b440bac4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:08:58.301160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71437d52-283d-42a5-b9ed-13b440bac4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:08:58.322600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-71437d52-283d-42a5-b9ed-13b440bac4a4 None None] MFA Rules not processed for user `4873db0f2fbf4a77a617f2102a5ccfae`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:08:58.323852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-71437d52-283d-42a5-b9ed-13b440bac4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:08:58.332335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1233/2448] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:08:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:06.767423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=19 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:07.071743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:07.084755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058fa15-6fc3-498e-9f84-1072d6070f3c service nova] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:07.091232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96e775-d935-4847-bc74-bb88ab8d673a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:07.092013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96e775-d935-4847-bc74-bb88ab8d673a service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:07.092360 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96e775-d935-4847-bc74-bb88ab8d673a service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:07.092701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96e775-d935-4847-bc74-bb88ab8d673a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:07.095670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2f96e775-d935-4847-bc74-bb88ab8d673a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:07.108844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96e775-d935-4847-bc74-bb88ab8d673a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:07.111437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96e775-d935-4847-bc74-bb88ab8d673a service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:07.123678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1216/2449] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:09:08.316515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:10.139560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71437d52-283d-42a5-b9ed-13b440bac4a4 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:10.154533 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71437d52-283d-42a5-b9ed-13b440bac4a4 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:10.161861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:10.162557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:10.162837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:10.163116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:10.165063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:10.179050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:10.180662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:10.192573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1234/2450] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:16.848760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005baf2-0897-4baf-ba83-2daaadc8f8b0 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:16.850552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005baf2-0897-4baf-ba83-2daaadc8f8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:16.850552 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005baf2-0897-4baf-ba83-2daaadc8f8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:16.871120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8005baf2-0897-4baf-ba83-2daaadc8f8b0 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:16.872451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8005baf2-0897-4baf-ba83-2daaadc8f8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:16.936616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1217/2451] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:09:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:16.949811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:16.951576 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:17.078287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d365-0b48-4ab3-a4be-9238e542e877 service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:17.086487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:17.087168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:17.087458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:17.087736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:17.102197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:17.103848 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:17.117896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1235/2452] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:09:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:09:26.929499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=31 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:27.112661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=38 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:28.808370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005baf2-0897-4baf-ba83-2daaadc8f8b0 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:28.819331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005baf2-0897-4baf-ba83-2daaadc8f8b0 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:28.823268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:28.824015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:28.824365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:28.824708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:28.826673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:28.844062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:28.845606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:28.856811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1218/2453] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:34.084902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:34.089119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.158502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:34.160917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.189563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadb6d8-96ef-41dc-927f-e2355b4c5379 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:34.199249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:34.200599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.201088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.201553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] PATH_INFO: `/v3/users/c70c44993ab240b1a5231df4c9d550c9` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.204741 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c70c44993ab240b1a5231df4c9d550c9)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:34.208398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:34.247490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c123df-9b4f-4795-b0bb-7d16576130b8 service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:34.262091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04721f2f-390f-4171-8c66-03758ec7e2c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:34.262780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04721f2f-390f-4171-8c66-03758ec7e2c0 service placement] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.263065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04721f2f-390f-4171-8c66-03758ec7e2c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.263342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04721f2f-390f-4171-8c66-03758ec7e2c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.267200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-04721f2f-390f-4171-8c66-03758ec7e2c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.280048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c70c44993ab240b1a5231df4c9d550c9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:34.281734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04721f2f-390f-4171-8c66-03758ec7e2c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:34.282817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c70c44993ab240b1a5231df4c9d550c9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:34.286157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04721f2f-390f-4171-8c66-03758ec7e2c0 service placement] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:34.295141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c70c44993ab240b1a5231df4c9d550c9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:34.296947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c70c44993ab240b1a5231df4c9d550c9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:34.301484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1236/2454] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:34.330294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9d3a2ec0-00a0-424b-9f18-71ed35f4bcfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c70c44993ab240b1a5231df4c9d550c9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:34.374593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1219/2455] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:09:34 2022] DELETE /identity/v3/users/c70c44993ab240b1a5231df4c9d550c9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:34.402959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81853286-724e-4d03-bc7d-be237c1635e3 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.403843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81853286-724e-4d03-bc7d-be237c1635e3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.404502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81853286-724e-4d03-bc7d-be237c1635e3 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.407481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1237/2456] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:09:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:09:34.414609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.414996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.415617 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.460746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:34.462947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.603891 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1220/2457] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:09:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:34.619774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81853286-724e-4d03-bc7d-be237c1635e3 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:34.622013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-81853286-724e-4d03-bc7d-be237c1635e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.716406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81853286-724e-4d03-bc7d-be237c1635e3 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:34.722034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:34.724700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.726383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05900b-f796-46ec-a1a9-d718649da2f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:34.727099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05900b-f796-46ec-a1a9-d718649da2f2 service glance] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.727455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05900b-f796-46ec-a1a9-d718649da2f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.727832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05900b-f796-46ec-a1a9-d718649da2f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.741954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05900b-f796-46ec-a1a9-d718649da2f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:34.743753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05900b-f796-46ec-a1a9-d718649da2f2 service glance] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:34.767228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1238/2458] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:34.843963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:34.851341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58faae3-c4d2-4ec4-86fb-cd3f23f92017 None None] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.851845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58faae3-c4d2-4ec4-86fb-cd3f23f92017 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.852342 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58faae3-c4d2-4ec4-86fb-cd3f23f92017 None None] PATH_INFO: `/` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.855687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1239/2459] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:09:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:09:34.862176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.862629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.863023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.881281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b552e6c-bd8b-4de9-8463-295df693ffd4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:34.883681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] MFA Rules not processed for user `45a558f9c7c84a05a90595fda7fccd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:34.885083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.891622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:34.893235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:34.893696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:34.894246 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] PATH_INFO: `/v3/projects/5e0c37d941434569b3dbd07e22125210` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:34.896932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e0c37d941434569b3dbd07e22125210)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:34.900456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:34.956089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1240/2460] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:09:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:34.974041 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:34.975203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e0c37d941434569b3dbd07e22125210', 'request_id': 'req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:34.975765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:34.986542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e0c37d941434569b3dbd07e22125210', 'request_id': 'req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:34.988611 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e56f7-c0ea-486d-82df-2364b356ed42 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:34.996048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e0c37d941434569b3dbd07e22125210', 'request_id': 'req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:35.012938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] The token cache is being invalidate because project 5e0c37d941434569b3dbd07e22125210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:09:35.013449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:35.015117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-bbfa52d3-d1fe-47bd-8524-c1464f238c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:35.017792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1221/2461] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:09:34 2022] DELETE /identity/v3/projects/5e0c37d941434569b3dbd07e22125210 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:09:35.026186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa979e-4ae9-4a4f-a321-cf316d2b7041 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.026651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa979e-4ae9-4a4f-a321-cf316d2b7041 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.026918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa979e-4ae9-4a4f-a321-cf316d2b7041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.055601 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9eaa979e-4ae9-4a4f-a321-cf316d2b7041 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:35.057068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9eaa979e-4ae9-4a4f-a321-cf316d2b7041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:35.060001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:35.075222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113b70c-d3f6-4eb0-8f92-c7468051afd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.076317 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b70c-d3f6-4eb0-8f92-c7468051afd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.077346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b70c-d3f6-4eb0-8f92-c7468051afd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.077952 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b70c-d3f6-4eb0-8f92-c7468051afd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.081186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f113b70c-d3f6-4eb0-8f92-c7468051afd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:35.094263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113b70c-d3f6-4eb0-8f92-c7468051afd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.096263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113b70c-d3f6-4eb0-8f92-c7468051afd6 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.107360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1241/2462] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:09:35.162094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1222/2463] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:35.171355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5015067-e05f-40a2-9eb9-e222f34e070a None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.172292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5015067-e05f-40a2-9eb9-e222f34e070a None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.173139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5015067-e05f-40a2-9eb9-e222f34e070a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.193972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d5015067-e05f-40a2-9eb9-e222f34e070a None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:35.196728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d5015067-e05f-40a2-9eb9-e222f34e070a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:35.211700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1242/2464] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:35.212949 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff476c6d-b62c-4fb6-bc08-aa6548874b19 None None] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.213467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff476c6d-b62c-4fb6-bc08-aa6548874b19 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.213898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff476c6d-b62c-4fb6-bc08-aa6548874b19 None None] PATH_INFO: `/` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.216015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1223/2465] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jun 29 08:09:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:09:35.216480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5015067-e05f-40a2-9eb9-e222f34e070a None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.224437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.225224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.225644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.229371 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5015067-e05f-40a2-9eb9-e222f34e070a None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.235489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.236609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.236894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.237388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.238147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:09:35.238449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.240561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.255320 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] MFA Rules not processed for user `a267b6f64f5f49b8b3f5837e701bc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:35.255747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1243/2466] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:09:35.256565 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:35.260431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.272663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448863e-fcfc-4fc5-89c8-6b316b100911 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.277582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.278252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.278541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.278830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.279520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.282292 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.339978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1244/2467] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/projects => generated 405 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:09:35.340284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1224/2468] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:35.342895 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.349811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.351329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:35.364685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab4431-5601-44ee-aa60-605467599689 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.377413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:35.423608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48684f1a-d045-49ab-aadc-14249f17d22c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.429246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.429956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.430242 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.430538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.431269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.433243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.433935 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.434754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.435011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.435340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.461071 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1245/2469] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:35.466840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.477954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8767cd7-046e-484f-958b-109de9d7d92b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.486076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.486774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.487057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.487336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.488087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.490329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.502814 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1246/2470] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:09:35.506874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.518774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aec2e8-ff27-4d56-8d13-39529196a329 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.522938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.523655 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c726c-ea17-47fd-8099-b222975c70f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.524327 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c726c-ea17-47fd-8099-b222975c70f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.524481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.524609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c726c-ea17-47fd-8099-b222975c70f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.524889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c726c-ea17-47fd-8099-b222975c70f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.525675 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c726c-ea17-47fd-8099-b222975c70f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.527332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c726c-ea17-47fd-8099-b222975c70f5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.538275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1225/2471] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:09:35.543150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1247/2472] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:09:35.544160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.555663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec648-77fb-4585-8ddb-9d5ee7180845 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.560309 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.561774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.562716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.563008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] PATH_INFO: `/v3/projects/85b1e6a9b8cf4c62aa9385e5805e1130/users/79448c3fc2db41399c7168223469c503/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.580430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b1e6a9b8cf4c62aa9385e5805e1130, user_id=79448c3fc2db41399c7168223469c503, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.583514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.599162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1226/2473] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:35 2022] PUT /identity/v3/projects/85b1e6a9b8cf4c62aa9385e5805e1130/users/79448c3fc2db41399c7168223469c503/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:09:35.609908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f108d6-b303-4739-a726-b37449d737e9 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.610200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f108d6-b303-4739-a726-b37449d737e9 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.610488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f108d6-b303-4739-a726-b37449d737e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.638760 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-d6f108d6-b303-4739-a726-b37449d737e9 None None] MFA Rules not processed for user `79448c3fc2db41399c7168223469c503`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:35.640500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-d6f108d6-b303-4739-a726-b37449d737e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:35.702686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:35.725110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1248/2474] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:35.734434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.842923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ace6e-e2b1-4582-9cfb-a9911d38d45d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.847983 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.848861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.849177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.849452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.850159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.853140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:35.885719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1227/2475] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/projects => generated 403 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:35.890715 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f108d6-b303-4739-a726-b37449d737e9 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:35.902012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f108d6-b303-4739-a726-b37449d737e9 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:35.974596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:35.975283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:35.975570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:35.975849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:35.976545 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:35.978504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:36.002799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1249/2476] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:09:35 2022] POST /identity/v3/users => generated 308 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:09:36.006880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:36.019812 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231503df-109f-41e5-a685-26ffde25b42f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:36.025124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:36.026425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.026723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.027032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.027756 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:36.029443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:36.041612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1228/2477] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:36.046013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:36.057793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e32d-e914-4483-bc96-81ed36dc311b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:36.064716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:36.065597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.066029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.066319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] PATH_INFO: `/v3/projects/d4cb3f9fb04b4f9ab916a2dfbaf85d19/users/50ea8885a9c0461bba42a0c2a3c7b172/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.086152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cb3f9fb04b4f9ab916a2dfbaf85d19, user_id=50ea8885a9c0461bba42a0c2a3c7b172, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:36.090301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:36.112395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1250/2478] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:36 2022] PUT /identity/v3/projects/d4cb3f9fb04b4f9ab916a2dfbaf85d19/users/50ea8885a9c0461bba42a0c2a3c7b172/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:36.116569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:36.130487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:36.140651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcbb0a-f910-4f20-b171-bfd3fef19f57 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:36.227452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b569e-c2d4-4b6e-bb81-f8589af1a0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:36.228146 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b569e-c2d4-4b6e-bb81-f8589af1a0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.228423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b569e-c2d4-4b6e-bb81-f8589af1a0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.228708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b569e-c2d4-4b6e-bb81-f8589af1a0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.229494 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b569e-c2d4-4b6e-bb81-f8589af1a0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:36.231386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b569e-c2d4-4b6e-bb81-f8589af1a0ad admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:36.249172 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1229/2479] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:36.251049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:36.264585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41464a68-56f8-447e-ab10-14311dfc7ea9 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:36.270069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:36.271061 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.271527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.271807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] PATH_INFO: `/v3/projects/d4cb3f9fb04b4f9ab916a2dfbaf85d19/users/50ea8885a9c0461bba42a0c2a3c7b172/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.273938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cb3f9fb04b4f9ab916a2dfbaf85d19, user_id=50ea8885a9c0461bba42a0c2a3c7b172, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:36.276806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:36.293544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1251/2480] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:36 2022] PUT /identity/v3/projects/d4cb3f9fb04b4f9ab916a2dfbaf85d19/users/50ea8885a9c0461bba42a0c2a3c7b172/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:36.306528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23e33c-9112-45e3-b34f-aefb15b764f5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.307005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23e33c-9112-45e3-b34f-aefb15b764f5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.307480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23e33c-9112-45e3-b34f-aefb15b764f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.353349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8d23e33c-9112-45e3-b34f-aefb15b764f5 None None] MFA Rules not processed for user `50ea8885a9c0461bba42a0c2a3c7b172`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:36.355341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8d23e33c-9112-45e3-b34f-aefb15b764f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:36.442966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1230/2481] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:36.458287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:36.461037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:36.535480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383efb14-828d-4e6f-8ede-db3f0c7c53f9 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:36.539474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4cf56-2afe-4822-b153-e0295fb1f4a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:36.540142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4cf56-2afe-4822-b153-e0295fb1f4a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.540430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4cf56-2afe-4822-b153-e0295fb1f4a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.540714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4cf56-2afe-4822-b153-e0295fb1f4a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.552795 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4cf56-2afe-4822-b153-e0295fb1f4a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:36.554279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4cf56-2afe-4822-b153-e0295fb1f4a0 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:36.566275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1252/2482] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:36.726894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23e33c-9112-45e3-b34f-aefb15b764f5 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:36.793298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23e33c-9112-45e3-b34f-aefb15b764f5 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:36.798451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:36.799348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.799631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.799907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] PATH_INFO: `/v3/users/79448c3fc2db41399c7168223469c503` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.801437 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79448c3fc2db41399c7168223469c503)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:36.803364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:36.835914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:36.855587 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79448c3fc2db41399c7168223469c503'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:36.856259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79448c3fc2db41399c7168223469c503'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:36.863924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79448c3fc2db41399c7168223469c503'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:36.864551 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79448c3fc2db41399c7168223469c503'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:36.877547 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79448c3fc2db41399c7168223469c503'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:36.902010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1231/2483] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:09:36 2022] DELETE /identity/v3/users/79448c3fc2db41399c7168223469c503 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:09:36.905112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:36.905511 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:36.905902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:36.924504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:36.926151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:36.976380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:36.996734 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1253/2484] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:37.015140 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:37.017134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:37.087993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ad29b-0e7b-4394-aaff-f3f23aa4b604 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:37.091609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22e353-a59c-47ec-b9d0-d109ec6ee842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:37.092825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22e353-a59c-47ec-b9d0-d109ec6ee842 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:37.093149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22e353-a59c-47ec-b9d0-d109ec6ee842 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:37.093428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22e353-a59c-47ec-b9d0-d109ec6ee842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:37.102768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22e353-a59c-47ec-b9d0-d109ec6ee842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:37.104362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22e353-a59c-47ec-b9d0-d109ec6ee842 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:37.114916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1232/2485] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:37.404129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:37.407000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:37.420300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c033126-3773-473e-a4f7-9c9fc6bd52bf None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:37.421240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c033126-3773-473e-a4f7-9c9fc6bd52bf None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:37.421240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c033126-3773-473e-a4f7-9c9fc6bd52bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:37.435652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e90d0-560b-4ee7-87f1-2f44621a264d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:37.440525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9c033126-3773-473e-a4f7-9c9fc6bd52bf None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:37.441760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9c033126-3773-473e-a4f7-9c9fc6bd52bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:37.441942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:37.442898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:37.443248 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:37.443595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] PATH_INFO: `/v3/projects/85b1e6a9b8cf4c62aa9385e5805e1130` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:37.445482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b1e6a9b8cf4c62aa9385e5805e1130)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:37.447931 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:37.500452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b1e6a9b8cf4c62aa9385e5805e1130', 'request_id': 'req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.506569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b1e6a9b8cf4c62aa9385e5805e1130', 'request_id': 'req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.512871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1233/2486] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:09:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:37.516596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b1e6a9b8cf4c62aa9385e5805e1130', 'request_id': 'req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.530910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] The token cache is being invalidate because project 85b1e6a9b8cf4c62aa9385e5805e1130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:09:37.531739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.532393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.538314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1254/2487] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:09:37 2022] DELETE /identity/v3/projects/85b1e6a9b8cf4c62aa9385e5805e1130 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:37.540503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:37.540859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c033126-3773-473e-a4f7-9c9fc6bd52bf None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:37.543168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:37.562853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c033126-3773-473e-a4f7-9c9fc6bd52bf None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:37.636562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:37.643182 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09b27e-ee9b-420c-a5ec-12764b88a2ba admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:37.649697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:37.650464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:37.650702 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:37.650960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:37.653546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:37.654157 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:37.655174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:37.655542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:37.655838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] PATH_INFO: `/v3/users/50ea8885a9c0461bba42a0c2a3c7b172` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:37.657170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ea8885a9c0461bba42a0c2a3c7b172)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:37.659111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:37.721192 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ea8885a9c0461bba42a0c2a3c7b172'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.721943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ea8885a9c0461bba42a0c2a3c7b172'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.733635 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ea8885a9c0461bba42a0c2a3c7b172'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.734525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ea8885a9c0461bba42a0c2a3c7b172'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.734808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:37.736416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:37.744945 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1255/2488] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:09:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:09:37.755300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-57a7aa05-d412-4ef4-a33d-7904106a5084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ea8885a9c0461bba42a0c2a3c7b172'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:37.770022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1234/2489] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:09:37 2022] DELETE /identity/v3/users/50ea8885a9c0461bba42a0c2a3c7b172 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:38.195115 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:38.197094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:38.219345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a33b69-224a-45f7-b0f4-7e3f7496ea14 service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:38.254645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:38.384930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:38.386421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:38.387003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:38.387369 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] PATH_INFO: `/v3/projects/d4cb3f9fb04b4f9ab916a2dfbaf85d19` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:38.390013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cb3f9fb04b4f9ab916a2dfbaf85d19)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:38.393541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:38.488663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cb3f9fb04b4f9ab916a2dfbaf85d19', 'request_id': 'req-a073aebf-052d-4a28-9c38-4396ade82d5d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:38.497209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cb3f9fb04b4f9ab916a2dfbaf85d19', 'request_id': 'req-a073aebf-052d-4a28-9c38-4396ade82d5d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:38.506417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cb3f9fb04b4f9ab916a2dfbaf85d19', 'request_id': 'req-a073aebf-052d-4a28-9c38-4396ade82d5d'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:38.522882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] The token cache is being invalidate because project d4cb3f9fb04b4f9ab916a2dfbaf85d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:09:38.523499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:38.533384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-a073aebf-052d-4a28-9c38-4396ade82d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:38.536205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1256/2490] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:09:38 2022] DELETE /identity/v3/projects/d4cb3f9fb04b4f9ab916a2dfbaf85d19 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:38.547667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe61cf0-d9ba-408c-95d0-809258e8d078 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:38.548259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe61cf0-d9ba-408c-95d0-809258e8d078 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:38.548834 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe61cf0-d9ba-408c-95d0-809258e8d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:38.585564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-fbe61cf0-d9ba-408c-95d0-809258e8d078 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:38.587250 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fbe61cf0-d9ba-408c-95d0-809258e8d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:38.668716 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1235/2491] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:38.681370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd78fc-1786-4a8b-98fd-d9e5e586b278 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:38.681833 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd78fc-1786-4a8b-98fd-d9e5e586b278 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:38.682347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd78fc-1786-4a8b-98fd-d9e5e586b278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:38.718338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ecbd78fc-1786-4a8b-98fd-d9e5e586b278 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:38.720059 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ecbd78fc-1786-4a8b-98fd-d9e5e586b278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:38.730376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1257/2492] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:38.735724 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe61cf0-d9ba-408c-95d0-809258e8d078 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:38.747416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe61cf0-d9ba-408c-95d0-809258e8d078 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:38.758315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:38.758968 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:38.759307 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:38.759652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:38.760503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:09:38.760825 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:38.773964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:38.801186 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1236/2493] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:09:38.803565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd78fc-1786-4a8b-98fd-d9e5e586b278 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:38.822605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd78fc-1786-4a8b-98fd-d9e5e586b278 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:38.830710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:38.830710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:38.830710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:38.830710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:38.831474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:38.832251 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:38.874603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1258/2494] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:09:38 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:38.884046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:38.914684 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050014e-917f-4548-8ffe-c9027284145e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.043567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.044418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.044706 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.045173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.047345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.051762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.064253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:39.101003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1237/2495] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:09:38 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:39.105090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.119431 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff82da2-2b42-4fc6-b30a-481cf3374140 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.125461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.126260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.126640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.126919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.127907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.129591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.152668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1259/2496] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:39.158954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.173107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d77bb7-2fca-45fd-87a1-36ee25918f1a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.179351 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a846800-c919-4103-9e6d-fccfc7d58b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.180450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846800-c919-4103-9e6d-fccfc7d58b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.180941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846800-c919-4103-9e6d-fccfc7d58b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.181479 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846800-c919-4103-9e6d-fccfc7d58b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.182599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a846800-c919-4103-9e6d-fccfc7d58b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.184417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a846800-c919-4103-9e6d-fccfc7d58b06 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.196818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1238/2497] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:39.204986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.219215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fc932-f0a5-4f6a-939b-b867d80c4173 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.225813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.226728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.227014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.227289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] PATH_INFO: `/v3/projects/6b9d4f40b43d40d59012f9284489fdc0/users/e66823c67bc24580806f55e69cbdb95c/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.250344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b9d4f40b43d40d59012f9284489fdc0, user_id=e66823c67bc24580806f55e69cbdb95c, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.253481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.277276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1260/2498] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:39 2022] PUT /identity/v3/projects/6b9d4f40b43d40d59012f9284489fdc0/users/e66823c67bc24580806f55e69cbdb95c/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:39.288570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935d56b-fb85-4bef-84f6-ab214171242a None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.289055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935d56b-fb85-4bef-84f6-ab214171242a None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.290036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935d56b-fb85-4bef-84f6-ab214171242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.359143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-d935d56b-fb85-4bef-84f6-ab214171242a None None] MFA Rules not processed for user `e66823c67bc24580806f55e69cbdb95c`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:39.361429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d935d56b-fb85-4bef-84f6-ab214171242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:39.477562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1239/2499] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:39.482458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.555063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:39.590485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ab697-834d-4605-bbfd-95490b01fe0c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.598018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.598879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.599271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.599724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.609214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.611184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.644152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1261/2500] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:09:39 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:39.649537 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935d56b-fb85-4bef-84f6-ab214171242a None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.665461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935d56b-fb85-4bef-84f6-ab214171242a None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.742763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.743463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.743841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.744534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.744824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.746821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.777876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1240/2501] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:09:39 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:09:39.780345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.788526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ada67f-2feb-434b-a8b2-0cd3b2e60f27 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.793763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.794467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.794777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.795035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.795896 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.797586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.820477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1262/2502] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:39.824535 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.837709 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ae693-9218-4a8d-bbd2-dce625bcd52b admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.843763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.845783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.846334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.846852 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] PATH_INFO: `/v3/projects/224633b8bea84301904edb949c9ba39a/users/f6fc2ba882944957b01944cc352659d1/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.868053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] RBAC: Authorizing `identity:create_grant(project_id=224633b8bea84301904edb949c9ba39a, user_id=f6fc2ba882944957b01944cc352659d1, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.870946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.888180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1241/2503] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:39 2022] PUT /identity/v3/projects/224633b8bea84301904edb949c9ba39a/users/f6fc2ba882944957b01944cc352659d1/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:39.893093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:39.905284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081d686-9d64-4f90-a3fa-55a31cfadad6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:39.970885 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c31ec-515d-48ba-88d6-c3d91828602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:39.971680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c31ec-515d-48ba-88d6-c3d91828602a admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:39.971973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c31ec-515d-48ba-88d6-c3d91828602a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:39.972311 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c31ec-515d-48ba-88d6-c3d91828602a admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:39.973526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c31ec-515d-48ba-88d6-c3d91828602a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:39.975393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c31ec-515d-48ba-88d6-c3d91828602a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:39.987891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1263/2504] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:39.992345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:40.006110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57f913-1478-4007-866c-2a443829154e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:40.012026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:40.013138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:40.013558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:40.014600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] PATH_INFO: `/v3/projects/224633b8bea84301904edb949c9ba39a/users/f6fc2ba882944957b01944cc352659d1/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:40.017381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=224633b8bea84301904edb949c9ba39a, user_id=f6fc2ba882944957b01944cc352659d1, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:40.020525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:40.032948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1242/2505] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:39 2022] PUT /identity/v3/projects/224633b8bea84301904edb949c9ba39a/users/f6fc2ba882944957b01944cc352659d1/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:09:40.046116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e5b3-507a-4a13-b7af-3dade675f268 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:40.046116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e5b3-507a-4a13-b7af-3dade675f268 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:40.046519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e5b3-507a-4a13-b7af-3dade675f268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:40.077445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9604e5b3-507a-4a13-b7af-3dade675f268 None None] MFA Rules not processed for user `f6fc2ba882944957b01944cc352659d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:40.078365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9604e5b3-507a-4a13-b7af-3dade675f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:40.198733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1264/2506] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:40.214766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:40.216864 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:40.263522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:40.312804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812534dd-698f-4225-a171-8b876fe7a50c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:40.317292 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:40.317868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:40.318156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:40.318424 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:40.328616 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:40.330118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:40.340840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1243/2507] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:40.445100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e5b3-507a-4a13-b7af-3dade675f268 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:40.462839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e5b3-507a-4a13-b7af-3dade675f268 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:40.501930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256a3b-befb-4320-91bc-209ddebadb17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:40.502627 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256a3b-befb-4320-91bc-209ddebadb17 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:40.502959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256a3b-befb-4320-91bc-209ddebadb17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:40.503223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256a3b-befb-4320-91bc-209ddebadb17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:40.550769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:40.594634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256a3b-befb-4320-91bc-209ddebadb17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:40.596527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256a3b-befb-4320-91bc-209ddebadb17 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:40.610412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1265/2508] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:50.335853 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=17 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:50.602304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=13 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:50.994649 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:51.078090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951460-d5f6-4277-9e04-40009206ebc0 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:51.083374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:51.084271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:51.084553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:51.084829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] PATH_INFO: `/v3/users/e66823c67bc24580806f55e69cbdb95c` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:51.086117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e66823c67bc24580806f55e69cbdb95c)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:51.088057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:51.145306 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e66823c67bc24580806f55e69cbdb95c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.146040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e66823c67bc24580806f55e69cbdb95c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.162647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e66823c67bc24580806f55e69cbdb95c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.163446 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e66823c67bc24580806f55e69cbdb95c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.180654 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e66823c67bc24580806f55e69cbdb95c'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.199613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1244/2509] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:09:50 2022] DELETE /identity/v3/users/e66823c67bc24580806f55e69cbdb95c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:51.205403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:51.205694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:51.205975 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:51.229455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:51.229455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:51.307625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1266/2510] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:51.317640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:51.319086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:51.391312 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2237b2-9211-495c-90ef-fe043d0ded1e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:51.395469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:51.396362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:51.396725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:51.397114 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:51.408961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:51.410940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:51.421704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1245/2511] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:51.812605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:51.814654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:51.833077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f121e82-61a6-4b54-b644-6bc6a3f49383 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:51.837729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:51.838598 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:51.838883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:51.839159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] PATH_INFO: `/v3/projects/6b9d4f40b43d40d59012f9284489fdc0` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:51.840346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b9d4f40b43d40d59012f9284489fdc0)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:51.842504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:51.903852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b9d4f40b43d40d59012f9284489fdc0', 'request_id': 'req-2cb274f7-35e9-46a1-a490-2cd0e020531e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.909530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b9d4f40b43d40d59012f9284489fdc0', 'request_id': 'req-2cb274f7-35e9-46a1-a490-2cd0e020531e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.917214 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b9d4f40b43d40d59012f9284489fdc0', 'request_id': 'req-2cb274f7-35e9-46a1-a490-2cd0e020531e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.929205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] The token cache is being invalidate because project 6b9d4f40b43d40d59012f9284489fdc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:09:51.929525 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.930180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:51.931987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1267/2512] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:09:51 2022] DELETE /identity/v3/projects/6b9d4f40b43d40d59012f9284489fdc0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:09:51.936334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:51.946442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3523fd-251f-4f09-95a5-ae54d697a1a1 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:51.965341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:52.007107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:52.008048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.008333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.008655 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] PATH_INFO: `/v3/users/f6fc2ba882944957b01944cc352659d1` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.009984 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6fc2ba882944957b01944cc352659d1)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:52.011985 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:52.059468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6fc2ba882944957b01944cc352659d1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.060134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6fc2ba882944957b01944cc352659d1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.067558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6fc2ba882944957b01944cc352659d1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.068219 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6fc2ba882944957b01944cc352659d1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.081007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-62a7c722-412a-476e-bf50-0df0ec605fa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6fc2ba882944957b01944cc352659d1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.095318 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1246/2513] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:09:51 2022] DELETE /identity/v3/users/f6fc2ba882944957b01944cc352659d1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:52.438979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:52.440594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:52.456635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb274f7-35e9-46a1-a490-2cd0e020531e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:52.518845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:52.519996 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.520273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.520547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] PATH_INFO: `/v3/projects/224633b8bea84301904edb949c9ba39a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.521781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224633b8bea84301904edb949c9ba39a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:52.523711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:52.560299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:52.580858 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224633b8bea84301904edb949c9ba39a', 'request_id': 'req-d380f842-02de-4a36-9dda-a62b3b479fa9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.586701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224633b8bea84301904edb949c9ba39a', 'request_id': 'req-d380f842-02de-4a36-9dda-a62b3b479fa9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.595907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224633b8bea84301904edb949c9ba39a', 'request_id': 'req-d380f842-02de-4a36-9dda-a62b3b479fa9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.610409 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] The token cache is being invalidate because project 224633b8bea84301904edb949c9ba39a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:09:52.610701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.611379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d380f842-02de-4a36-9dda-a62b3b479fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:52.613246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1268/2514] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:09:52 2022] DELETE /identity/v3/projects/224633b8bea84301904edb949c9ba39a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:52.618247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc5300-312f-4ebe-8168-9ee83958bd2e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.618808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc5300-312f-4ebe-8168-9ee83958bd2e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.618808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc5300-312f-4ebe-8168-9ee83958bd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.646792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-eadc5300-312f-4ebe-8168-9ee83958bd2e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:52.648818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-eadc5300-312f-4ebe-8168-9ee83958bd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:52.730630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1247/2515] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:52.738695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468fe60-ac68-47ff-9dce-c5e2240c340d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.738984 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468fe60-ac68-47ff-9dce-c5e2240c340d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.739268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468fe60-ac68-47ff-9dce-c5e2240c340d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.759957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-4468fe60-ac68-47ff-9dce-c5e2240c340d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:52.761241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4468fe60-ac68-47ff-9dce-c5e2240c340d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:52.772058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1269/2516] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:52.776042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc5300-312f-4ebe-8168-9ee83958bd2e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:52.786350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc5300-312f-4ebe-8168-9ee83958bd2e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:52.791253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:52.791921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.792208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.792490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.793267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:09:52.793577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:52.795308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:52.809104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1248/2517] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:09:52.813466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468fe60-ac68-47ff-9dce-c5e2240c340d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:52.825360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468fe60-ac68-47ff-9dce-c5e2240c340d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:52.832750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de60731-215a-4eba-aa75-fe7a1a270919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:52.833364 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de60731-215a-4eba-aa75-fe7a1a270919 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.833545 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de60731-215a-4eba-aa75-fe7a1a270919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.833822 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de60731-215a-4eba-aa75-fe7a1a270919 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.834687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de60731-215a-4eba-aa75-fe7a1a270919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:52.836447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de60731-215a-4eba-aa75-fe7a1a270919 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:52.876050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1270/2518] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:09:52 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:52.877507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:52.890865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9cecc-319a-4987-b7a3-78696f9e045a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:52.898954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72d318-9639-419b-b114-a9582dd0b040 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.899787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72d318-9639-419b-b114-a9582dd0b040 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.900285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72d318-9639-419b-b114-a9582dd0b040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.923654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-dd72d318-9639-419b-b114-a9582dd0b040 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:52.925691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-dd72d318-9639-419b-b114-a9582dd0b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:52.938171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:52.990784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:52.991727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:52.991857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:52.992190 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:52.993033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:52.994954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.010215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1271/2519] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jun 29 08:09:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:53.024259 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1249/2520] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:09:52 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:53.028378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72d318-9639-419b-b114-a9582dd0b040 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.029877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.032034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:53.042001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72d318-9639-419b-b114-a9582dd0b040 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.048816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.049573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.049887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.050129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.051322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.053154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.070554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1272/2521] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:09:53.071772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.081891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a3f98-6f5d-4501-85da-fe4a3f306888 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.086847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.087534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.087803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.088076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.088802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.090464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.104432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1273/2522] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:09:53.105522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.116155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea175959-3324-48e1-b359-76418dba6345 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.123666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d302e1-7d3a-4721-bc31-bfc352f01b0d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.124439 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.125406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.125688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.125967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] PATH_INFO: `/v3/projects/a8f57b1d64b64ad88b82e6737354148a/users/fd21ff6127874d50adee286b16c35488/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.128966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e5858-ed4a-4033-a237-cf3c15c4e10d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.129689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e5858-ed4a-4033-a237-cf3c15c4e10d service swift] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.129975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e5858-ed4a-4033-a237-cf3c15c4e10d service swift] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.130257 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e5858-ed4a-4033-a237-cf3c15c4e10d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.139595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e5858-ed4a-4033-a237-cf3c15c4e10d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.141130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e5858-ed4a-4033-a237-cf3c15c4e10d service swift] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.143426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f57b1d64b64ad88b82e6737354148a, user_id=fd21ff6127874d50adee286b16c35488, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.146284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.150367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1250/2523] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:09:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:09:53.161888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1274/2524] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:53 2022] PUT /identity/v3/projects/a8f57b1d64b64ad88b82e6737354148a/users/fd21ff6127874d50adee286b16c35488/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:53.175055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9407e4-71af-4d75-ada9-9c8e45edb6a7 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.176116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9407e4-71af-4d75-ada9-9c8e45edb6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.176619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9407e4-71af-4d75-ada9-9c8e45edb6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.224727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2a9407e4-71af-4d75-ada9-9c8e45edb6a7 None None] MFA Rules not processed for user `fd21ff6127874d50adee286b16c35488`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:53.226043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2a9407e4-71af-4d75-ada9-9c8e45edb6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:53.333702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1251/2525] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:53.338070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.400130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:53.432096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea69e30-220f-4357-b170-0b2228503294 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.448939 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.449957 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.450238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.452390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.453419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.458165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.525063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1275/2526] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:09:53 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:09:53.527971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9407e4-71af-4d75-ada9-9c8e45edb6a7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.548208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9407e4-71af-4d75-ada9-9c8e45edb6a7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.598460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:53.628375 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.629279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.629796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.630160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.630944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.632950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.678209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.690560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fe8ed-ebd1-4666-bd43-d65cb9d1b52f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.691295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1252/2527] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:09:53 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:09:53.696479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.697284 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.697763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.698111 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.698924 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.700630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.717842 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1276/2528] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:53.719576 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.731455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd50b5-653f-4c60-bd2c-3718b9a618c9 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.739917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.740847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.741167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.741580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] PATH_INFO: `/v3/projects/001a03dbba274df8b7c4ef00b8200bac/users/c850774d72b847d7a9832c513cd49b30/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.758558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=001a03dbba274df8b7c4ef00b8200bac, user_id=c850774d72b847d7a9832c513cd49b30, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.761304 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.780508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1253/2529] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:53 2022] PUT /identity/v3/projects/001a03dbba274df8b7c4ef00b8200bac/users/c850774d72b847d7a9832c513cd49b30/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:53.784852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.795948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7d3f-676b-42bc-9989-8fdcbb97a0ec admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.872818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7ddca-12c1-43d0-82ac-a7f98469afff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.874042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7ddca-12c1-43d0-82ac-a7f98469afff admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.877383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7ddca-12c1-43d0-82ac-a7f98469afff admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.879031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7ddca-12c1-43d0-82ac-a7f98469afff admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.880942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7ddca-12c1-43d0-82ac-a7f98469afff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.883927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7ddca-12c1-43d0-82ac-a7f98469afff admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.898540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1277/2530] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:53.902789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:53.917041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac38ac-7b4e-4368-a018-46339a69e38c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:53.924047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:53.925038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.925581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.925870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] PATH_INFO: `/v3/projects/001a03dbba274df8b7c4ef00b8200bac/users/c850774d72b847d7a9832c513cd49b30/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:53.928054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=001a03dbba274df8b7c4ef00b8200bac, user_id=c850774d72b847d7a9832c513cd49b30, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:53.930789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:53.947227 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1254/2531] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:53 2022] PUT /identity/v3/projects/001a03dbba274df8b7c4ef00b8200bac/users/c850774d72b847d7a9832c513cd49b30/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:09:53.973050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9000a0-5f56-4715-827e-b561e6b3b92f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:53.973462 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9000a0-5f56-4715-827e-b561e6b3b92f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:53.974601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9000a0-5f56-4715-827e-b561e6b3b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:54.011194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7c9000a0-5f56-4715-827e-b561e6b3b92f None None] MFA Rules not processed for user `c850774d72b847d7a9832c513cd49b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:54.012352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7c9000a0-5f56-4715-827e-b561e6b3b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:54.098864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1278/2532] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:54.114371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:54.116791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:54.196469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d61f0b-4c8b-4e30-bf3b-2147221bba3e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:54.200297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:54.201661 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:54.202042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:54.202390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:54.212840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:54.217808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:54.229030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1255/2533] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:54.336471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9000a0-5f56-4715-827e-b561e6b3b92f None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:54.346200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9000a0-5f56-4715-827e-b561e6b3b92f None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:54.349826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ce236-35a6-4027-a25e-949defccaafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:54.350464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ce236-35a6-4027-a25e-949defccaafd service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:54.350746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ce236-35a6-4027-a25e-949defccaafd service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:54.351026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ce236-35a6-4027-a25e-949defccaafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:54.390410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:54.410753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ce236-35a6-4027-a25e-949defccaafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:54.412252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ce236-35a6-4027-a25e-949defccaafd service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:54.422374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1279/2534] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:09:57.103303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:57.169363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba7b43-8f52-4ac4-9e4a-f7d9efa2ada1 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:57.174598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:57.175758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:57.176042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:57.176316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] PATH_INFO: `/v3/users/fd21ff6127874d50adee286b16c35488` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:57.177559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd21ff6127874d50adee286b16c35488)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:57.179525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:57.222126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:57.236014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd21ff6127874d50adee286b16c35488'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:57.236883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd21ff6127874d50adee286b16c35488'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:57.244338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd21ff6127874d50adee286b16c35488'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:57.244686 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd21ff6127874d50adee286b16c35488'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:57.258833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd21ff6127874d50adee286b16c35488'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:57.275827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1256/2535] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:09:57 2022] DELETE /identity/v3/users/fd21ff6127874d50adee286b16c35488 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:57.281716 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:57.282449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:57.282449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:57.304127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:57.305425 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:57.377782 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1280/2536] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:57.388402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:57.390420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:57.463989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac08f79-f924-465b-954e-24ce28aca2bb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:57.469943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:57.470883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:57.471276 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:57.471636 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:57.491384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:57.491384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:57.503237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1257/2537] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:09:57.918376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:57.920637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:57.944718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6b776-a583-4a94-9f2e-f982ba010839 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:57.953168 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:57.954687 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:57.955199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:57.955707 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] PATH_INFO: `/v3/projects/a8f57b1d64b64ad88b82e6737354148a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:57.958003 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f57b1d64b64ad88b82e6737354148a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:57.962390 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:58.027735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f57b1d64b64ad88b82e6737354148a', 'request_id': 'req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.034630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f57b1d64b64ad88b82e6737354148a', 'request_id': 'req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.042598 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f57b1d64b64ad88b82e6737354148a', 'request_id': 'req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.055692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] The token cache is being invalidate because project a8f57b1d64b64ad88b82e6737354148a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:09:58.055980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.056636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.058225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1281/2538] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:09:57 2022] DELETE /identity/v3/projects/a8f57b1d64b64ad88b82e6737354148a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:58.063641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:58.081797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee11f8-7374-4dbd-9b4f-2a0180cfb3fa service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:58.150180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:58.151067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:58.151355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:58.151631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] PATH_INFO: `/v3/users/c850774d72b847d7a9832c513cd49b30` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:58.152838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c850774d72b847d7a9832c513cd49b30)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:58.155393 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:58.209638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c850774d72b847d7a9832c513cd49b30'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.210300 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c850774d72b847d7a9832c513cd49b30'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.218646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c850774d72b847d7a9832c513cd49b30'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.219381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c850774d72b847d7a9832c513cd49b30'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.220749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:58.241348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-358e577f-f66f-4060-8531-7389d85ecffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c850774d72b847d7a9832c513cd49b30'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:58.256610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1258/2539] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:09:58 2022] DELETE /identity/v3/users/c850774d72b847d7a9832c513cd49b30 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:58.992283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:58.993845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:59.021568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da3eb0-9051-4e70-bbe6-a682acd3c5c5 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.085872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.087038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.087173 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.087454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] PATH_INFO: `/v3/projects/001a03dbba274df8b7c4ef00b8200bac` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.088863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=001a03dbba274df8b7c4ef00b8200bac)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.090794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.094383 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:59.152138 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001a03dbba274df8b7c4ef00b8200bac', 'request_id': 'req-7ab1d28e-c4b8-4b47-997d-797afafb67fa'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:59.158331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001a03dbba274df8b7c4ef00b8200bac', 'request_id': 'req-7ab1d28e-c4b8-4b47-997d-797afafb67fa'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:59.166290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001a03dbba274df8b7c4ef00b8200bac', 'request_id': 'req-7ab1d28e-c4b8-4b47-997d-797afafb67fa'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:59.178628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] The token cache is being invalidate because project 001a03dbba274df8b7c4ef00b8200bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:09:59.178922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:59.179646 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-7ab1d28e-c4b8-4b47-997d-797afafb67fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:09:59.181844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1282/2540] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:09:58 2022] DELETE /identity/v3/projects/001a03dbba274df8b7c4ef00b8200bac => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:59.187746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344bb55-c990-4a1f-8a83-07fafe46827f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.188287 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344bb55-c990-4a1f-8a83-07fafe46827f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.188403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344bb55-c990-4a1f-8a83-07fafe46827f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.207979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e344bb55-c990-4a1f-8a83-07fafe46827f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:59.209282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e344bb55-c990-4a1f-8a83-07fafe46827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:59.278906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1259/2541] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:59.287555 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de3c03-2a56-4b79-a0fa-6ec65d4f383d None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.287844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de3c03-2a56-4b79-a0fa-6ec65d4f383d None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.288196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de3c03-2a56-4b79-a0fa-6ec65d4f383d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.307358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f1de3c03-2a56-4b79-a0fa-6ec65d4f383d None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:59.308770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f1de3c03-2a56-4b79-a0fa-6ec65d4f383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:59.317105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1283/2542] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:59.321463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344bb55-c990-4a1f-8a83-07fafe46827f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:59.332487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344bb55-c990-4a1f-8a83-07fafe46827f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.338549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.339228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.339517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.339798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.340545 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:09:59.340860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.343042 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.359295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1260/2543] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:09:59.360379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de3c03-2a56-4b79-a0fa-6ec65d4f383d None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:59.370104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de3c03-2a56-4b79-a0fa-6ec65d4f383d None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.375089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc5c99-fc9a-491f-9f55-45c5d45512d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.375765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5c99-fc9a-491f-9f55-45c5d45512d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.376044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5c99-fc9a-491f-9f55-45c5d45512d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.376321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5c99-fc9a-491f-9f55-45c5d45512d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.377129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc5c99-fc9a-491f-9f55-45c5d45512d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.378985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc5c99-fc9a-491f-9f55-45c5d45512d6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.408301 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1284/2544] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:09:59 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:09:59.413769 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:59.425026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55784327-7595-4a3e-8e98-4e4530123ab7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.481678 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f83a1-9d7e-44e3-a2d4-bc35c8cf8d93 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.481969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f83a1-9d7e-44e3-a2d4-bc35c8cf8d93 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.482246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f83a1-9d7e-44e3-a2d4-bc35c8cf8d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.505906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.506192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-450f83a1-9d7e-44e3-a2d4-bc35c8cf8d93 None None] MFA Rules not processed for user `4873db0f2fbf4a77a617f2102a5ccfae`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:59.506842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.507207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.507456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-450f83a1-9d7e-44e3-a2d4-bc35c8cf8d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:59.507634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.508415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.510726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.550967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1261/2545] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:09:59 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:09:59.552899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:59.568517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003eb0d-e435-41e8-9f05-bb2720571797 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.573618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.574299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.574588 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.574865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.575598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.577813 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.582466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1285/2546] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:09:59.594821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1262/2547] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:09:59.599295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f83a1-9d7e-44e3-a2d4-bc35c8cf8d93 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:59.611237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f83a1-9d7e-44e3-a2d4-bc35c8cf8d93 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.615932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57ae1f-408c-4405-b119-879d5bc44dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.616604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57ae1f-408c-4405-b119-879d5bc44dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.616891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57ae1f-408c-4405-b119-879d5bc44dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.617208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57ae1f-408c-4405-b119-879d5bc44dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.617943 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57ae1f-408c-4405-b119-879d5bc44dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.619544 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57ae1f-408c-4405-b119-879d5bc44dd8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.630858 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1286/2548] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:09:59.635072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:59.646358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367f96a-14f6-4923-b9ed-daa453376812 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.657803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.657803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.657803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.657803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] PATH_INFO: `/v3/projects/941e14f704b74aa7afe5ef3a75a3f24b/users/b887b672801d436a80fde4c5b2f8ad61/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.682241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] RBAC: Authorizing `identity:create_grant(project_id=941e14f704b74aa7afe5ef3a75a3f24b, user_id=b887b672801d436a80fde4c5b2f8ad61, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.685050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.708546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1263/2549] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:09:59 2022] PUT /identity/v3/projects/941e14f704b74aa7afe5ef3a75a3f24b/users/b887b672801d436a80fde4c5b2f8ad61/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:09:59.713828 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd4e03-9402-4940-8210-6ad088bada97 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.714123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd4e03-9402-4940-8210-6ad088bada97 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.714407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd4e03-9402-4940-8210-6ad088bada97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.734019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-43dd4e03-9402-4940-8210-6ad088bada97 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:09:59.735332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-43dd4e03-9402-4940-8210-6ad088bada97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:59.799014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:59.818212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1287/2550] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:09:59.828839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:09:59.830568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:09:59.850151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:09:59.896560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21f203-17f8-42fd-8f87-727ecc6d926f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:09:59.900745 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:09:59.901453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:09:59.901746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:09:59.902031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:09:59.913048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:09:59.914669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:09:59.926116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1264/2551] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:10:00.735653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617b0e0-272e-49d0-8eba-d80700946fdd None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:00.736190 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617b0e0-272e-49d0-8eba-d80700946fdd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:00.736351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617b0e0-272e-49d0-8eba-d80700946fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:00.764195 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5617b0e0-272e-49d0-8eba-d80700946fdd None None] MFA Rules not processed for user `b887b672801d436a80fde4c5b2f8ad61`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:00.765469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5617b0e0-272e-49d0-8eba-d80700946fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:00.854778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1288/2552] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:00.867961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:00.869560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:00.944798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9e8ce-bd88-4be7-bf6c-1ab50472cf66 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:00.949866 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5a750-395c-4fa7-8107-ce9c264db1c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:00.950620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5a750-395c-4fa7-8107-ce9c264db1c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:00.950904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5a750-395c-4fa7-8107-ce9c264db1c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:00.951766 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5a750-395c-4fa7-8107-ce9c264db1c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:00.963255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5a750-395c-4fa7-8107-ce9c264db1c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:00.964791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5a750-395c-4fa7-8107-ce9c264db1c5 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:00.976096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1265/2553] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:10:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:10:02.337207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac7fd-1ee4-497a-b0c5-0903d885cedb None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:02.337785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac7fd-1ee4-497a-b0c5-0903d885cedb None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:02.337903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac7fd-1ee4-497a-b0c5-0903d885cedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:02.359050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-286ac7fd-1ee4-497a-b0c5-0903d885cedb None None] MFA Rules not processed for user `b887b672801d436a80fde4c5b2f8ad61`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:02.360257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-286ac7fd-1ee4-497a-b0c5-0903d885cedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:02.374040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1289/2554] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:10.971181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=45 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:12.352608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=36 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:19.058908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:19.059452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:19.059452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:19.083802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] MFA Rules not processed for user `b887b672801d436a80fde4c5b2f8ad61`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:19.084759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:19.092107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1266/2555] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:29.078210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:40.527826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:40.528722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:40.528722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:40.553728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:40.555087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:40.651491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1290/2556] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:10:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:40.683509 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:40.684625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:40.783459 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da254c0-0c4a-460c-9763-d70774b0a4db None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:40.789273 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:40.790054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:40.790241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:40.791078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:40.801567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:40.803333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:40.814098 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1267/2557] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:10:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:10:42.115476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:42.119558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:42.150022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f790-1a46-4595-ad33-d2c2c442fc05 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:42.160703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:42.162469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:42.163362 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:42.164094 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:42.167963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:42.188997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:42.190772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:42.204471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1291/2558] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:10:46.190915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:46.192805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:46.223387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fe94-d5e5-486a-ae5d-d406b6ac759f service swift] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:46.228641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:46.230034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:46.230319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:46.230599 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] PATH_INFO: `/v3/users/c9065575ed314f6f8f3915bf596911e5` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:46.232096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9065575ed314f6f8f3915bf596911e5)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:46.234738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:46.310673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9065575ed314f6f8f3915bf596911e5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.311553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9065575ed314f6f8f3915bf596911e5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.321041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9065575ed314f6f8f3915bf596911e5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.322089 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9065575ed314f6f8f3915bf596911e5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.340876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a41f300-ed59-4549-8774-a6f48d37f0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9065575ed314f6f8f3915bf596911e5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.364877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1268/2559] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:10:46 2022] DELETE /identity/v3/users/c9065575ed314f6f8f3915bf596911e5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:10:46.639454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:46.654615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5654f28-ba75-4277-87bb-52f6bea44874 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:46.772798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:46.774452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:46.774985 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:46.775865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] PATH_INFO: `/v3/projects/4cd09897be9c4642b91dfb53a536e7d1` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:46.778660 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd09897be9c4642b91dfb53a536e7d1)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:46.783273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:46.884756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd09897be9c4642b91dfb53a536e7d1', 'request_id': 'req-28c4352b-3d76-4c78-a395-787cf4b48d4b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.894141 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd09897be9c4642b91dfb53a536e7d1', 'request_id': 'req-28c4352b-3d76-4c78-a395-787cf4b48d4b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.909564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd09897be9c4642b91dfb53a536e7d1', 'request_id': 'req-28c4352b-3d76-4c78-a395-787cf4b48d4b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.930378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] The token cache is being invalidate because project 4cd09897be9c4642b91dfb53a536e7d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:10:46.931811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.933912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-28c4352b-3d76-4c78-a395-787cf4b48d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:46.942488 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1292/2560] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:10:46 2022] DELETE /identity/v3/projects/4cd09897be9c4642b91dfb53a536e7d1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:10:46.946279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80834-9910-40a8-a6f6-c3bde7373194 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:46.946874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80834-9910-40a8-a6f6-c3bde7373194 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:46.948428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80834-9910-40a8-a6f6-c3bde7373194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:46.987148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-dbd80834-9910-40a8-a6f6-c3bde7373194 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:46.988879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-dbd80834-9910-40a8-a6f6-c3bde7373194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:47.125542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1269/2561] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:47.136264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0785e4-2967-4022-93b7-916bdf5866f7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.145378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0785e4-2967-4022-93b7-916bdf5866f7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.145731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0785e4-2967-4022-93b7-916bdf5866f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.176130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2d0785e4-2967-4022-93b7-916bdf5866f7 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:47.181088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2d0785e4-2967-4022-93b7-916bdf5866f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:47.205353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1293/2562] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:10:47.229014 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd80834-9910-40a8-a6f6-c3bde7373194 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:47.234797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:47.243454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd80834-9910-40a8-a6f6-c3bde7373194 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:47.251632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:47.252670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.253388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.253761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.254523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:10:47.254839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:47.256691 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:47.276285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1270/2563] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:10:47.277401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0785e4-2967-4022-93b7-916bdf5866f7 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:47.298351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0785e4-2967-4022-93b7-916bdf5866f7 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:47.304170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:47.304846 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.305874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.305874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.306123 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:47.308194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:47.335504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:47.361260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1294/2564] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:10:47 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:10:47.367689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:47.380549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ca8a9-3bda-4d2e-9165-c3696f0c59b3 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:47.473997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:47.474838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.475132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.475558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.476355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:47.478468 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:47.512997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1271/2565] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:10:47 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:10:47.514279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:47.529480 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f93c9-eb46-499d-a8b2-c7847f56938c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:47.537402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:47.538637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.538923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.539368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.540632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:47.542412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:47.565023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1295/2566] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:10:47.566776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:47.577744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd946f4-4bb3-4607-9e70-d0ca19bb0585 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:47.585637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125a59e-ae19-4d09-98f0-f164f7d61bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:47.586442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125a59e-ae19-4d09-98f0-f164f7d61bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.586754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125a59e-ae19-4d09-98f0-f164f7d61bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.587004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125a59e-ae19-4d09-98f0-f164f7d61bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.587842 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125a59e-ae19-4d09-98f0-f164f7d61bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:47.589564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125a59e-ae19-4d09-98f0-f164f7d61bc8 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:47.604230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1272/2567] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:10:47.608239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:47.620569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4ed4-cc6c-453e-8202-4f1459f688df admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:47.627490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:47.628654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.629095 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.629345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] PATH_INFO: `/v3/projects/deded3a6edb749019760ffda53379320/users/799e595fee724c088c8c1bb1242e80b3/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.648148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=deded3a6edb749019760ffda53379320, user_id=799e595fee724c088c8c1bb1242e80b3, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:47.651105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:47.684732 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1296/2568] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:10:47 2022] PUT /identity/v3/projects/deded3a6edb749019760ffda53379320/users/799e595fee724c088c8c1bb1242e80b3/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:10:47.696311 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38f3b-90fa-449b-a080-dfb58893df1d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.696722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38f3b-90fa-449b-a080-dfb58893df1d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.697207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38f3b-90fa-449b-a080-dfb58893df1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.739012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1be38f3b-90fa-449b-a080-dfb58893df1d None None] MFA Rules not processed for user `799e595fee724c088c8c1bb1242e80b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:47.740272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1be38f3b-90fa-449b-a080-dfb58893df1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:47.848811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1273/2569] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:47.849991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:47.851923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:47.929093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adf232-3ebe-4fa7-9a51-55d422488d0a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:47.934250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db77f2-1c0a-47ff-a390-ed530a48354a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:47.935298 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db77f2-1c0a-47ff-a390-ed530a48354a service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:47.935807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db77f2-1c0a-47ff-a390-ed530a48354a service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:47.936386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db77f2-1c0a-47ff-a390-ed530a48354a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:47.950446 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db77f2-1c0a-47ff-a390-ed530a48354a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:47.953167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db77f2-1c0a-47ff-a390-ed530a48354a service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:47.974735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1297/2570] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:10:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:10:57.284442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38f3b-90fa-449b-a080-dfb58893df1d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:57.384558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:57.393369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38f3b-90fa-449b-a080-dfb58893df1d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:57.400610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:57.402168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:57.402719 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:57.403210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] PATH_INFO: `/v3/users/799e595fee724c088c8c1bb1242e80b3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:57.404903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=799e595fee724c088c8c1bb1242e80b3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:57.407910 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:57.468149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '799e595fee724c088c8c1bb1242e80b3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:57.469673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '799e595fee724c088c8c1bb1242e80b3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:57.478929 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '799e595fee724c088c8c1bb1242e80b3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:57.484597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '799e595fee724c088c8c1bb1242e80b3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:57.507430 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '799e595fee724c088c8c1bb1242e80b3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:57.525810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1274/2571] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:10:57 2022] DELETE /identity/v3/users/799e595fee724c088c8c1bb1242e80b3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:10:57.531157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:57.531631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:57.531866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:57.549748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:57.551661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:57.617963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1298/2572] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:10:57.627788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:57.629239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:57.689242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cda73e-c338-4b04-b0a2-700b3a481b6c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:57.692816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ec533-45ad-4c1f-832d-d130329bb445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:57.693491 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ec533-45ad-4c1f-832d-d130329bb445 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:57.693779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ec533-45ad-4c1f-832d-d130329bb445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:57.694057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ec533-45ad-4c1f-832d-d130329bb445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:57.703611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ec533-45ad-4c1f-832d-d130329bb445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:57.705160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ec533-45ad-4c1f-832d-d130329bb445 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:57.714681 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1275/2573] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:10:57.996735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.000010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:58.016640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572f344-8e15-48fd-8648-6ff0af2ccdc8 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.021348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.022205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.022483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.022754 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] PATH_INFO: `/v3/projects/deded3a6edb749019760ffda53379320` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.023971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deded3a6edb749019760ffda53379320)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.025979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.074218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deded3a6edb749019760ffda53379320', 'request_id': 'req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:58.079352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:58.080564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deded3a6edb749019760ffda53379320', 'request_id': 'req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:58.089600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deded3a6edb749019760ffda53379320', 'request_id': 'req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:58.101585 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] The token cache is being invalidate because project deded3a6edb749019760ffda53379320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:10:58.101871 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:58.102548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ed9cd457-9fec-4734-b755-9d9f0bab8bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:10:58.104082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1299/2574] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:10:57 2022] DELETE /identity/v3/projects/deded3a6edb749019760ffda53379320 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:10:58.109332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb676d7-c7c0-4489-a25e-ccbcf2619690 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.109618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb676d7-c7c0-4489-a25e-ccbcf2619690 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.109916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb676d7-c7c0-4489-a25e-ccbcf2619690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.129466 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-bfb676d7-c7c0-4489-a25e-ccbcf2619690 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:58.130534 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-bfb676d7-c7c0-4489-a25e-ccbcf2619690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:58.199647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1276/2575] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:10:58.210454 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7451fe-693e-4e12-96e9-9064bc0f2c07 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.210744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7451fe-693e-4e12-96e9-9064bc0f2c07 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.211026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7451fe-693e-4e12-96e9-9064bc0f2c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.236347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2f7451fe-693e-4e12-96e9-9064bc0f2c07 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:58.237719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2f7451fe-693e-4e12-96e9-9064bc0f2c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:58.245998 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1300/2576] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:10:58.251563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb676d7-c7c0-4489-a25e-ccbcf2619690 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.262105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb676d7-c7c0-4489-a25e-ccbcf2619690 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.268992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.269680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.269964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.270241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.270978 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:10:58.271285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.272892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.286814 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1277/2577] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:10:58.291661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7451fe-693e-4e12-96e9-9064bc0f2c07 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.303824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7451fe-693e-4e12-96e9-9064bc0f2c07 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.308834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543ad59-98c0-447c-9364-0f1e16229122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.309554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543ad59-98c0-447c-9364-0f1e16229122 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.309837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543ad59-98c0-447c-9364-0f1e16229122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.310115 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543ad59-98c0-447c-9364-0f1e16229122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.310873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543ad59-98c0-447c-9364-0f1e16229122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.313737 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543ad59-98c0-447c-9364-0f1e16229122 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.350510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1301/2578] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:10:58.352779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.366244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e6d54-ac23-40bc-8f99-48ecc58cc1d8 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.370580 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:58.441040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.441720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.442029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.442294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.443010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.445076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.455722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.456644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.457392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.472409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1278/2579] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:10:58.473797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.485030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbd24d-2b50-4aa7-aeae-6d230e9b354f admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.486811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:58.488726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:58.491106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.491863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.492132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.492416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.493202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.494827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.512294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1279/2580] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:10:58.513533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.524135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad60925-b1a5-4fb0-abfe-1921c3816393 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.530204 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.530892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.531342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.531621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.532395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.534032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.544863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1280/2581] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:10:58.549903 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.558025 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1302/2582] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:58.560677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a690b-f967-4acb-b11c-1556df156004 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.567426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867e930-6c8d-475b-b3f5-a335bb93856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.567829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e930-6c8d-475b-b3f5-a335bb93856f admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.568117 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e930-6c8d-475b-b3f5-a335bb93856f admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.568676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e930-6c8d-475b-b3f5-a335bb93856f admin admin] PATH_INFO: `/v3/projects/51f39799892849ae9daaaaaf7639e03c/users/cbd90ae1b4164c208ce236aa2857235a/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.571807 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.573605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:58.586235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867e930-6c8d-475b-b3f5-a335bb93856f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f39799892849ae9daaaaaf7639e03c, user_id=cbd90ae1b4164c208ce236aa2857235a, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.589289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867e930-6c8d-475b-b3f5-a335bb93856f admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.604329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1281/2583] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:10:58 2022] PUT /identity/v3/projects/51f39799892849ae9daaaaaf7639e03c/users/cbd90ae1b4164c208ce236aa2857235a/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:10:58.617871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3ea1-8505-460e-abf7-26f4f66f119c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.618963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3ea1-8505-460e-abf7-26f4f66f119c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.618963 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3ea1-8505-460e-abf7-26f4f66f119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.652220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-82ec3ea1-8505-460e-abf7-26f4f66f119c None None] MFA Rules not processed for user `cbd90ae1b4164c208ce236aa2857235a`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:58.654046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-82ec3ea1-8505-460e-abf7-26f4f66f119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:58.656104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073dcef-967f-44f7-a30b-40dcf622239c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.743918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1282/2584] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:10:58.744775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec3ea1-8505-460e-abf7-26f4f66f119c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.773498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.774244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.774550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.775324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.779806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:58.818094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:58.839818 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec3ea1-8505-460e-abf7-26f4f66f119c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.847476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.848609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.849128 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.850015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.851540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.853510 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.855106 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.855243 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:58.864755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1303/2585] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:10:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 08:10:58.895367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1283/2586] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:10:58.899826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:58.911058 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f8306-ddd6-4661-946c-348b830559a4 service swift] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:58.982711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:58.984057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:58.984346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:58.984633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:58.985498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:58.987552 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.013626 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1304/2587] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:10:58 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:10:59.017726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.029249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34085696-e455-498e-a136-21aacc4ca81a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:59.036039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:59.036784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.037166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.037508 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.038523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:59.040297 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.056970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1284/2588] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:10:59.058099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.068436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037948a-0cb4-449b-ad6c-7d323df904e6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:59.074068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:59.075139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.075244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.075506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] PATH_INFO: `/v3/projects/0c605bd3d83d40a99b1050fa891dd7f8/users/14039cd9cfbb45199edac1f30d9d7c3e/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.092106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c605bd3d83d40a99b1050fa891dd7f8, user_id=14039cd9cfbb45199edac1f30d9d7c3e, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:59.095427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.110337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1305/2589] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:10:59 2022] PUT /identity/v3/projects/0c605bd3d83d40a99b1050fa891dd7f8/users/14039cd9cfbb45199edac1f30d9d7c3e/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:10:59.114210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.125669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8519bd2-f7b5-484a-b34f-dbccae130666 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:59.191685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b615f-be0d-491a-afa8-6ee6a0bd07be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:59.191685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b615f-be0d-491a-afa8-6ee6a0bd07be admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.191685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b615f-be0d-491a-afa8-6ee6a0bd07be admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.192177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b615f-be0d-491a-afa8-6ee6a0bd07be admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.192739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b615f-be0d-491a-afa8-6ee6a0bd07be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:59.194532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b615f-be0d-491a-afa8-6ee6a0bd07be admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.206704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1285/2590] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:10:59.210994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.224307 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231a9d4-26a6-453b-884b-cf1dd176c02a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:59.229689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:59.230586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.231337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.231337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] PATH_INFO: `/v3/projects/0c605bd3d83d40a99b1050fa891dd7f8/users/14039cd9cfbb45199edac1f30d9d7c3e/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.234079 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c605bd3d83d40a99b1050fa891dd7f8, user_id=14039cd9cfbb45199edac1f30d9d7c3e, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:59.236566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.247977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1306/2591] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:10:59 2022] PUT /identity/v3/projects/0c605bd3d83d40a99b1050fa891dd7f8/users/14039cd9cfbb45199edac1f30d9d7c3e/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:10:59.259167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8ec22-fb81-4742-a291-15228aca07d8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.259480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8ec22-fb81-4742-a291-15228aca07d8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.259746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8ec22-fb81-4742-a291-15228aca07d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.291880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-fff8ec22-fb81-4742-a291-15228aca07d8 None None] MFA Rules not processed for user `14039cd9cfbb45199edac1f30d9d7c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:10:59.293215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fff8ec22-fb81-4742-a291-15228aca07d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:59.378398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1286/2592] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:10:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:10:59.391762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.393513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:10:59.453253 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1c8d2-93d3-4ef5-a0ed-80873287cdb9 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:59.456867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:59.457559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.457844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.458120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.467200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:59.469448 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.474581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:10:59.481050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1307/2593] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:10:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:10:59.831158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8ec22-fb81-4742-a291-15228aca07d8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.848298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8ec22-fb81-4742-a291-15228aca07d8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:59.852957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14039cd9cfbb45199edac1f30d9d7c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c605bd3d83d40a99b1050fa891dd7f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:59.853666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.853950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.854230 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.854973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:10:59.855383 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:59.857215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.870972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1287/2594] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:10:59.878157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.894357 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269182a-a918-4703-8225-28e3268b5de6 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:10:59.901313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14039cd9cfbb45199edac1f30d9d7c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c605bd3d83d40a99b1050fa891dd7f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:10:59.902099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:10:59.902389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:10:59.902674 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:10:59.903381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:10:59.905379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:10:59.946238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1308/2595] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:10:59 2022] POST /identity/v3/projects => generated 355 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:10:59.950229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:10:59.972049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1bb2-8880-4594-88a9-6084b3f2284d tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:00.012156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:00.072164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14039cd9cfbb45199edac1f30d9d7c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c605bd3d83d40a99b1050fa891dd7f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:00.072906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:00.073281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:00.073612 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:00.074760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:11:00.075029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:00.083512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:00.107614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1288/2596] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:11:00.163902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:00.177009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21384e-aff6-4862-a46d-c619c9bd3ed1 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:00.186982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14039cd9cfbb45199edac1f30d9d7c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c605bd3d83d40a99b1050fa891dd7f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:00.188319 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:00.188819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:00.189421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] PATH_INFO: `/v3/projects/a6e75e2cb4af4816a369e720fc3872b3` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:00.192263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a6e75e2cb4af4816a369e720fc3872b3)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:00.195179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:00.260845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e75e2cb4af4816a369e720fc3872b3', 'request_id': 'req-df5bb448-1163-4e13-9c12-2cc7794e7b82'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.267553 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e75e2cb4af4816a369e720fc3872b3', 'request_id': 'req-df5bb448-1163-4e13-9c12-2cc7794e7b82'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.277466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e75e2cb4af4816a369e720fc3872b3', 'request_id': 'req-df5bb448-1163-4e13-9c12-2cc7794e7b82'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.295176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] The token cache is being invalidate because project a6e75e2cb4af4816a369e720fc3872b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:00.295392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.296347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-df5bb448-1163-4e13-9c12-2cc7794e7b82 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.298455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1309/2597] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:00 2022] DELETE /identity/v3/projects/a6e75e2cb4af4816a369e720fc3872b3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:00.544266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:00.546202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:00.660575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ad69-f811-42f4-abc5-31f09ca28563 tempest-VolumeQuotaClassesTest-263357751 tempest-VolumeQuotaClassesTest-263357751-project-admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:00.736824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:00.737860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:00.738255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:00.738609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] PATH_INFO: `/v3/users/cbd90ae1b4164c208ce236aa2857235a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:00.739916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd90ae1b4164c208ce236aa2857235a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:00.742141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:00.804707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd90ae1b4164c208ce236aa2857235a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.805678 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd90ae1b4164c208ce236aa2857235a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.815302 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd90ae1b4164c208ce236aa2857235a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.816002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd90ae1b4164c208ce236aa2857235a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.825666 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:00.834593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd90ae1b4164c208ce236aa2857235a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:00.849525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1289/2598] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:00 2022] DELETE /identity/v3/users/cbd90ae1b4164c208ce236aa2857235a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:00.853981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:00.854377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:00.854605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:00.877045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:00.878290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:00.922798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:00.925056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:00.956708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1310/2599] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:00.973608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:00.976873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:01.021435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd62fcc-e612-4746-aaf2-637e61ba4193 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:01.026877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:01.027558 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:01.027843 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:01.028122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:01.030628 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:01.042470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:01.044540 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:01.051614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59994cf2-1bce-4e5c-b0a0-2cd25f42c7ae None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:01.055603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1290/2600] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:11:01.055869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:01.056416 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:01.056554 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:01.056824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:01.057211 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:01.058067 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:01.069452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:01.071436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:01.088607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1311/2601] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:01.088723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982447ee-4ac9-4b63-8a12-58b81648da16 service placement] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:01.093735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:01.094698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:01.095083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:01.095238 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] PATH_INFO: `/v3/users/b887b672801d436a80fde4c5b2f8ad61` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:01.098247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b887b672801d436a80fde4c5b2f8ad61)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:01.100232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:01.180528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b887b672801d436a80fde4c5b2f8ad61'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.180826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b887b672801d436a80fde4c5b2f8ad61'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.194395 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b887b672801d436a80fde4c5b2f8ad61'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.195699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b887b672801d436a80fde4c5b2f8ad61'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.213380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b887b672801d436a80fde4c5b2f8ad61'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.232624 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1291/2602] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:01 2022] DELETE /identity/v3/users/b887b672801d436a80fde4c5b2f8ad61 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:01.543857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:01.556087 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:01.562049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f3490-427c-4aa3-b7b9-98328e5c2f01 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:01.650101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:01.650969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:01.651257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:01.651530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] PATH_INFO: `/v3/projects/941e14f704b74aa7afe5ef3a75a3f24b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:01.652750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=941e14f704b74aa7afe5ef3a75a3f24b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:01.654757 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:01.702923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:01.704480 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:01.717086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '941e14f704b74aa7afe5ef3a75a3f24b', 'request_id': 'req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.725467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '941e14f704b74aa7afe5ef3a75a3f24b', 'request_id': 'req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.729209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35afb8a-b773-481d-a5d2-55f1cce1c873 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:01.735616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '941e14f704b74aa7afe5ef3a75a3f24b', 'request_id': 'req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.753430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] The token cache is being invalidate because project 941e14f704b74aa7afe5ef3a75a3f24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:01.753729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.754412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-2b3632ee-cd14-4a54-b24b-fee8ab2c4895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.760503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1312/2603] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:01 2022] DELETE /identity/v3/projects/941e14f704b74aa7afe5ef3a75a3f24b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:01.770761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba29eeb-37bf-4160-8f09-df9d5f5410b8 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:01.771091 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba29eeb-37bf-4160-8f09-df9d5f5410b8 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:01.771326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba29eeb-37bf-4160-8f09-df9d5f5410b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:01.788299 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:01.797987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fba29eeb-37bf-4160-8f09-df9d5f5410b8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:01.799634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fba29eeb-37bf-4160-8f09-df9d5f5410b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:01.811916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:01.812744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:01.813167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:01.813541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] PATH_INFO: `/v3/projects/51f39799892849ae9daaaaaf7639e03c` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:01.814901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f39799892849ae9daaaaaf7639e03c)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:01.816966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:01.871710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f39799892849ae9daaaaaf7639e03c', 'request_id': 'req-edcae721-7d91-4809-929e-ae79d60e2da7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.876151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1313/2604] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:01.880202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f39799892849ae9daaaaaf7639e03c', 'request_id': 'req-edcae721-7d91-4809-929e-ae79d60e2da7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.884328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22e4e-4a34-458b-9f81-731f2e795332 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:01.884870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22e4e-4a34-458b-9f81-731f2e795332 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:01.885121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22e4e-4a34-458b-9f81-731f2e795332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:01.891319 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f39799892849ae9daaaaaf7639e03c', 'request_id': 'req-edcae721-7d91-4809-929e-ae79d60e2da7'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.907070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] The token cache is being invalidate because project 51f39799892849ae9daaaaaf7639e03c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:01.907513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.908315 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:01.910263 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1292/2605] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:01 2022] DELETE /identity/v3/projects/51f39799892849ae9daaaaaf7639e03c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:01.913699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:01.913830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-12f22e4e-4a34-458b-9f81-731f2e795332 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:01.915054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:01.915412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-12f22e4e-4a34-458b-9f81-731f2e795332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:01.961485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:02.002161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcae721-7d91-4809-929e-ae79d60e2da7 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.005451 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1314/2606] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:02.007236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.011458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22e4e-4a34-458b-9f81-731f2e795332 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.013828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.014915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.015210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] PATH_INFO: `/v3/users/14039cd9cfbb45199edac1f30d9d7c3e` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.017466 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14039cd9cfbb45199edac1f30d9d7c3e)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.020615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.021940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22e4e-4a34-458b-9f81-731f2e795332 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.026938 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.027599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.027889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.028167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.029069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:11:02.029379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.031170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.048619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1315/2607] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:11:02.049808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.060775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14290c72-7d81-495a-aea1-26efa5f4107f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.071154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.071820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.072112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.072398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.073129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.075069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.086802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14039cd9cfbb45199edac1f30d9d7c3e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.087800 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14039cd9cfbb45199edac1f30d9d7c3e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.095763 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14039cd9cfbb45199edac1f30d9d7c3e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.096634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14039cd9cfbb45199edac1f30d9d7c3e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.105063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1316/2608] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:02 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:02.109230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.110875 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:02.111005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14039cd9cfbb45199edac1f30d9d7c3e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.114239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:02.132943 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1293/2609] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:01 2022] DELETE /identity/v3/users/14039cd9cfbb45199edac1f30d9d7c3e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:02.197286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889daa3-9f35-4867-bc84-a7f6c701fbfa admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.268466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.268466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.268466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.268466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.268948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.282613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.333024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1317/2610] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:02 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:02.336167 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.347409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f2fd-4901-48a4-a545-826df4348428 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.353996 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.354680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.354972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.355255 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.355999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.357660 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.372091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1294/2611] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:02.374775 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.386961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985831d-9a2a-4806-a892-091014ba26b1 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.392049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.392745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.393084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.393367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.394097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.395697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.406564 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1318/2612] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:02.410838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.422038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924e1f2-03ab-4605-8910-767c79fec531 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.427613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aea5fc-84ae-4f87-b17e-4ad889554d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.428597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aea5fc-84ae-4f87-b17e-4ad889554d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.428957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aea5fc-84ae-4f87-b17e-4ad889554d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.429402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aea5fc-84ae-4f87-b17e-4ad889554d5b admin admin] PATH_INFO: `/v3/projects/a2dc1a0a6ac041449241778f044e21ce/users/ec6639b5bf58466a945b080bd820dde9/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.446960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aea5fc-84ae-4f87-b17e-4ad889554d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2dc1a0a6ac041449241778f044e21ce, user_id=ec6639b5bf58466a945b080bd820dde9, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.448960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.450504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aea5fc-84ae-4f87-b17e-4ad889554d5b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.451067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:02.466676 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1295/2613] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:02 2022] PUT /identity/v3/projects/a2dc1a0a6ac041449241778f044e21ce/users/ec6639b5bf58466a945b080bd820dde9/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:02.467183 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fd79f-cf50-4843-a4c1-f118a7e6df7f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.478096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44675-30fb-407f-80da-139bf814f57e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.478391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44675-30fb-407f-80da-139bf814f57e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.478669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44675-30fb-407f-80da-139bf814f57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.505049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:02.508717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-95b44675-30fb-407f-80da-139bf814f57e None None] MFA Rules not processed for user `ec6639b5bf58466a945b080bd820dde9`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:02.510159 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-95b44675-30fb-407f-80da-139bf814f57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:02.540224 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.541125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.541412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.541692 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] PATH_INFO: `/v3/projects/0c605bd3d83d40a99b1050fa891dd7f8` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.542906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c605bd3d83d40a99b1050fa891dd7f8)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.544812 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.585056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1296/2614] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:02.591224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44675-30fb-407f-80da-139bf814f57e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.593556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-95b44675-30fb-407f-80da-139bf814f57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:02.595134 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c605bd3d83d40a99b1050fa891dd7f8', 'request_id': 'req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.600825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c605bd3d83d40a99b1050fa891dd7f8', 'request_id': 'req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.610042 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c605bd3d83d40a99b1050fa891dd7f8', 'request_id': 'req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.622391 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] The token cache is being invalidate because project 0c605bd3d83d40a99b1050fa891dd7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:02.622688 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.623353 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-52b88e38-f4f0-4c62-9f0f-0415fcef2fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:02.625404 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1319/2615] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:02 2022] DELETE /identity/v3/projects/0c605bd3d83d40a99b1050fa891dd7f8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:02.633642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09365e-5513-4806-9b60-11168c9c82e4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.633936 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09365e-5513-4806-9b60-11168c9c82e4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.634227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09365e-5513-4806-9b60-11168c9c82e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.651861 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-cf09365e-5513-4806-9b60-11168c9c82e4 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:02.653009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-cf09365e-5513-4806-9b60-11168c9c82e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:02.658126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44675-30fb-407f-80da-139bf814f57e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.727971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.729095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.729095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.729539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.731638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:02.737837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1320/2616] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:02.745731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282aa33-adeb-4bc6-9f96-b05d3f2fec44 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.746110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282aa33-adeb-4bc6-9f96-b05d3f2fec44 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.746396 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282aa33-adeb-4bc6-9f96-b05d3f2fec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.757274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:02.766824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-7282aa33-adeb-4bc6-9f96-b05d3f2fec44 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:02.767933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7282aa33-adeb-4bc6-9f96-b05d3f2fec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:02.785816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1321/2617] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:02.791313 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7282aa33-adeb-4bc6-9f96-b05d3f2fec44 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.803186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7282aa33-adeb-4bc6-9f96-b05d3f2fec44 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.810285 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.810967 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.811381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.812309 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.813093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:11:02.813407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.815050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.825514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.827837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.828151 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1322/2618] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:11:02.832724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.838890 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1297/2619] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:02.845281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe429931-64f2-4ee4-afb4-5b4db8ee271b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.850470 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.851375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.851603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.851970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.852926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.856125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:02.894158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1323/2620] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:02 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:02.898671 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:02.911364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cd9ed-fa86-4a03-bd16-a17a1e7175da service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:02.983048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:02.983811 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:02.984180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:02.984403 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:02.985138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:02.987450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.018918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1298/2621] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:02 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:11:03.020928 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.038007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9a27-a14a-42bd-b590-dbe90a3d4dfb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.048113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.049338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.049338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.049338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.049338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.053999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.071786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1324/2622] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:03.077923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.090476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324659e8-b65a-443a-b0e5-fb3f1fa8ba0d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.095258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdac348-4f70-4b6a-958a-80090f5dd983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.097039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdac348-4f70-4b6a-958a-80090f5dd983 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.097262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdac348-4f70-4b6a-958a-80090f5dd983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.097593 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdac348-4f70-4b6a-958a-80090f5dd983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.098760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdac348-4f70-4b6a-958a-80090f5dd983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.100492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdac348-4f70-4b6a-958a-80090f5dd983 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.125110 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1299/2623] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:03.132139 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.145393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:03.151232 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3bc50-e515-4171-b4e1-d75673a5f1fe admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.159288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.160530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.161043 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.161729 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] PATH_INFO: `/v3/projects/79fc4acea7f14cd8a70e7699328034ef/users/175cfea5fec444eaae7f8f54273bdc15/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.180872 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79fc4acea7f14cd8a70e7699328034ef, user_id=175cfea5fec444eaae7f8f54273bdc15, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.183762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.207694 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1325/2624] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:03 2022] PUT /identity/v3/projects/79fc4acea7f14cd8a70e7699328034ef/users/175cfea5fec444eaae7f8f54273bdc15/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:03.219642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae35c1-6c23-4173-a6b8-0b9048449b4d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.220008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae35c1-6c23-4173-a6b8-0b9048449b4d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.220830 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae35c1-6c23-4173-a6b8-0b9048449b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.251301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-35ae35c1-6c23-4173-a6b8-0b9048449b4d None None] MFA Rules not processed for user `175cfea5fec444eaae7f8f54273bdc15`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:03.252626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-35ae35c1-6c23-4173-a6b8-0b9048449b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:03.329791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1300/2625] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:03.334786 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.400192 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6a38e-b466-49dd-aab9-798a6e96d5b2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.405773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.406447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.406730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.407007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.407850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.409784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.438946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1326/2626] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:03 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:03.443072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae35c1-6c23-4173-a6b8-0b9048449b4d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.455181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae35c1-6c23-4173-a6b8-0b9048449b4d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.482164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:03.517981 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.518744 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.518999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.519248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.520078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.522072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.545019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1301/2627] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:03 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:03.548056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.558718 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29425a-0de9-4d2a-826a-06aff928eafe admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.565531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.566574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.566937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.567218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.568029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.569750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.581334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1327/2628] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:03.585712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.597057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfceaf8a-24ff-4d07-a126-2510890f1bc9 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.601896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984850f8-9591-4302-972a-b152be79395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.602801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984850f8-9591-4302-972a-b152be79395c admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.603100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984850f8-9591-4302-972a-b152be79395c admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.603355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984850f8-9591-4302-972a-b152be79395c admin admin] PATH_INFO: `/v3/projects/eb053c69ef2d4514b371f3d37e937584/users/9ce392da1f9647119f36ace67d07547b/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.620075 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984850f8-9591-4302-972a-b152be79395c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb053c69ef2d4514b371f3d37e937584, user_id=9ce392da1f9647119f36ace67d07547b, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.623470 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984850f8-9591-4302-972a-b152be79395c admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.641861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1302/2629] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:03 2022] PUT /identity/v3/projects/eb053c69ef2d4514b371f3d37e937584/users/9ce392da1f9647119f36ace67d07547b/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:03.645816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.657161 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc42bc5-a312-4ff0-ae7d-75e6355e58a7 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.720209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452f8d2-671e-475a-aa96-0245f33bc8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.720879 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f8d2-671e-475a-aa96-0245f33bc8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.721216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f8d2-671e-475a-aa96-0245f33bc8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.721497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f8d2-671e-475a-aa96-0245f33bc8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.722222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452f8d2-671e-475a-aa96-0245f33bc8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.723821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452f8d2-671e-475a-aa96-0245f33bc8d2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.734442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1328/2630] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:03.738801 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984850f8-9591-4302-972a-b152be79395c admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.749410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984850f8-9591-4302-972a-b152be79395c admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.754032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.754947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.755183 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.755456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] PATH_INFO: `/v3/projects/eb053c69ef2d4514b371f3d37e937584/users/9ce392da1f9647119f36ace67d07547b/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.757432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb053c69ef2d4514b371f3d37e937584, user_id=9ce392da1f9647119f36ace67d07547b, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.760049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.771040 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1303/2631] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:03 2022] PUT /identity/v3/projects/eb053c69ef2d4514b371f3d37e937584/users/9ce392da1f9647119f36ace67d07547b/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:03.783777 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.784068 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.784360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.813099 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] MFA Rules not processed for user `9ce392da1f9647119f36ace67d07547b`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:03.814333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:03.858073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:03.886322 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1329/2632] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:11:03.897454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:03.899173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:03.957848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df59a6-3a98-4fc8-8696-e3bd35c40aa2 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:03.961368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:03.962012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:03.962296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:03.962573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:03.972361 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:03.973795 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:03.983201 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1304/2633] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:04.028720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:04.030550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:04.042419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d2c8d-c7e6-4da9-a8c6-1da278c4af83 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:04.046125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a6101-e64b-41e2-b193-4995433edfaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:04.046756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a6101-e64b-41e2-b193-4995433edfaf service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:04.047063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a6101-e64b-41e2-b193-4995433edfaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:04.047333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a6101-e64b-41e2-b193-4995433edfaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:04.103722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a6101-e64b-41e2-b193-4995433edfaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:04.105178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a6101-e64b-41e2-b193-4995433edfaf service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:04.114269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1330/2634] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:08.575174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:08.754702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3e268-4ae3-4513-9cb5-32c8736cf287 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:08.776222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:08.777859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:08.778214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:08.778836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] PATH_INFO: `/v3/users/175cfea5fec444eaae7f8f54273bdc15` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:08.782394 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=175cfea5fec444eaae7f8f54273bdc15)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:08.785877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:08.836282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:08.893965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175cfea5fec444eaae7f8f54273bdc15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:08.895731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175cfea5fec444eaae7f8f54273bdc15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:08.910349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175cfea5fec444eaae7f8f54273bdc15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:08.911225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175cfea5fec444eaae7f8f54273bdc15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:08.929816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175cfea5fec444eaae7f8f54273bdc15'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:08.948453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1305/2635] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:08 2022] DELETE /identity/v3/users/175cfea5fec444eaae7f8f54273bdc15 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:08.954888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:08.955758 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:08.956125 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:08.979783 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:08.981738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:09.056400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:09.058224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:09.064199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1331/2636] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:09.079009 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:09.080821 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:09.159286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da328-6415-4263-9fea-418b173868b4 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:09.167024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:09.167701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:09.168268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:09.168557 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:09.171513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:09.180104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f85f3-764d-43fa-ac42-65eca36a34ce None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:09.186972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:09.187434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:09.187799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:09.188075 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:09.188339 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:09.189095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:09.202205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1306/2637] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:11:09.203328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:09.205043 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:09.219865 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1332/2638] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:11:09.578317 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:09.580135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:09.598698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab11cd3-f93c-4202-a64a-85c7f99d8eb6 service placement] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:09.603561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:09.604425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:09.604708 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:09.605005 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] PATH_INFO: `/v3/projects/79fc4acea7f14cd8a70e7699328034ef` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:09.606247 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79fc4acea7f14cd8a70e7699328034ef)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:09.608953 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:09.667387 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79fc4acea7f14cd8a70e7699328034ef', 'request_id': 'req-4c1a99ee-46fa-4168-b50e-b783ddda0844'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.674595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79fc4acea7f14cd8a70e7699328034ef', 'request_id': 'req-4c1a99ee-46fa-4168-b50e-b783ddda0844'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.681819 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79fc4acea7f14cd8a70e7699328034ef', 'request_id': 'req-4c1a99ee-46fa-4168-b50e-b783ddda0844'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.696116 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] The token cache is being invalidate because project 79fc4acea7f14cd8a70e7699328034ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:09.696426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.697090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.698768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1307/2639] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:09 2022] DELETE /identity/v3/projects/79fc4acea7f14cd8a70e7699328034ef => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:09.703909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:09.714240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9d3e-4b43-430b-96ae-f8eea6b6cfad service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:09.728624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:09.780743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:09.781640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:09.781954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:09.782354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] PATH_INFO: `/v3/users/9ce392da1f9647119f36ace67d07547b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:09.783560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce392da1f9647119f36ace67d07547b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:09.785556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:09.843730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce392da1f9647119f36ace67d07547b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.860001 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce392da1f9647119f36ace67d07547b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.860849 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce392da1f9647119f36ace67d07547b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.869731 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce392da1f9647119f36ace67d07547b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.870336 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-531c1ae5-48dd-44a9-91ac-67bd27e51269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce392da1f9647119f36ace67d07547b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:09.885361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1333/2640] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:09 2022] DELETE /identity/v3/users/9ce392da1f9647119f36ace67d07547b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:10.246346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:10.249209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:10.269471 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a99ee-46fa-4168-b50e-b783ddda0844 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:10.282539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:10.340149 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:10.341047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.341335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.341610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] PATH_INFO: `/v3/projects/eb053c69ef2d4514b371f3d37e937584` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.342875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb053c69ef2d4514b371f3d37e937584)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:10.345008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:10.397256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb053c69ef2d4514b371f3d37e937584', 'request_id': 'req-a896498b-5169-4906-9368-13164f5f4190'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:10.403252 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb053c69ef2d4514b371f3d37e937584', 'request_id': 'req-a896498b-5169-4906-9368-13164f5f4190'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:10.418776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb053c69ef2d4514b371f3d37e937584', 'request_id': 'req-a896498b-5169-4906-9368-13164f5f4190'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:10.435839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] The token cache is being invalidate because project eb053c69ef2d4514b371f3d37e937584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:10.436856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:10.437453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-a896498b-5169-4906-9368-13164f5f4190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:10.438855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1308/2641] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:10 2022] DELETE /identity/v3/projects/eb053c69ef2d4514b371f3d37e937584 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:10.444089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f848-e9b8-48b0-9d4d-0d18d215ffcd None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.444601 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f848-e9b8-48b0-9d4d-0d18d215ffcd None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.444696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f848-e9b8-48b0-9d4d-0d18d215ffcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.463433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-9ce3f848-e9b8-48b0-9d4d-0d18d215ffcd None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:10.464647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9ce3f848-e9b8-48b0-9d4d-0d18d215ffcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:10.533236 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1334/2642] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:10.540955 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115f03-877e-4684-9733-b371e98aa96c None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.541268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115f03-877e-4684-9733-b371e98aa96c None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.541546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115f03-877e-4684-9733-b371e98aa96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.561656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-42115f03-877e-4684-9733-b371e98aa96c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:10.563019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-42115f03-877e-4684-9733-b371e98aa96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:10.571547 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1309/2643] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:11:10.575619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f848-e9b8-48b0-9d4d-0d18d215ffcd None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:10.586315 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f848-e9b8-48b0-9d4d-0d18d215ffcd None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:10.595474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:10.595474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.595474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.595474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.597519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:11:10.597519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:10.598228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:10.613027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1335/2644] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:11:10.617781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115f03-877e-4684-9733-b371e98aa96c None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:10.631453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115f03-877e-4684-9733-b371e98aa96c None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:10.636785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:10.637674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.638120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.638539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.639536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:10.641828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:10.672293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1310/2645] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:10 2022] POST /identity/v3/projects => generated 409 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:10.684568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:10.699103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f510d4d-8ba2-4adf-9ac0-d61c480e5ae6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:10.725568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:10.769264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:10.769959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.770283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.770569 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.771306 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:10.773256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:10.801853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1336/2646] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:10 2022] POST /identity/v3/users => generated 305 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:10.802467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:10.814522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40985af4-f5b9-4f2c-88f6-11dd12bc34a1 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:10.820863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:10.821605 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.821893 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.822170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.822908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:10.824626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:10.838477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1311/2647] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:10.845199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:10.856589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9509e13-033d-459b-9f0b-e990ef3b24dc admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:10.862432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c780ae-7517-4e7b-8082-3fd25f0be3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:10.863154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c780ae-7517-4e7b-8082-3fd25f0be3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.863445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c780ae-7517-4e7b-8082-3fd25f0be3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.863726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c780ae-7517-4e7b-8082-3fd25f0be3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.864463 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c780ae-7517-4e7b-8082-3fd25f0be3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:10.866112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c780ae-7517-4e7b-8082-3fd25f0be3fd admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:10.880606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1337/2648] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:10.882156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:10.893280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35733d3-59bc-4235-8285-f6dad3384970 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:10.897977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:10.898881 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.899160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.899438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] PATH_INFO: `/v3/projects/7d92fab7a6c54f71a75db3f2e7236fc2/users/8814041ab313448ca967886388ee53ae/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.916578 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d92fab7a6c54f71a75db3f2e7236fc2, user_id=8814041ab313448ca967886388ee53ae, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:10.919877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:10.935425 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1312/2649] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:10 2022] PUT /identity/v3/projects/7d92fab7a6c54f71a75db3f2e7236fc2/users/8814041ab313448ca967886388ee53ae/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:10.946568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65820-9d39-4ce9-89f8-cf362f3ee38e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:10.946854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65820-9d39-4ce9-89f8-cf362f3ee38e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:10.947131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65820-9d39-4ce9-89f8-cf362f3ee38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:10.978481 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-52d65820-9d39-4ce9-89f8-cf362f3ee38e None None] MFA Rules not processed for user `8814041ab313448ca967886388ee53ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:10.979720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-52d65820-9d39-4ce9-89f8-cf362f3ee38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:11.057400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1338/2650] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:11.062734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:11.119750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:11.154399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46759a26-2fdd-45d3-a07e-245eb9c05366 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:11.160614 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:11.161322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.161613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.161892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.162591 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:11.164511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:11.197735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1313/2651] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:11 2022] POST /identity/v3/projects => generated 407 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:11.199505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65820-9d39-4ce9-89f8-cf362f3ee38e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:11.213331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65820-9d39-4ce9-89f8-cf362f3ee38e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:11.286178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:11.286910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.287196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.287495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.288203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:11.290246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:11.313013 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1339/2652] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:11 2022] POST /identity/v3/users => generated 310 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:11.317203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:11.329378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71257-8aa8-4660-ae72-57c2ad08d0a9 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:11.334349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:11.335133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.335483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.335833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.336669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:11.338484 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:11.351457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1314/2653] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:11.355933 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:11.367636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfa21c-49a7-455c-93f2-571191ca62ac admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:11.372485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:11.373535 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.374402 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.374700 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] PATH_INFO: `/v3/projects/9c3f161655e74064a3565a298e35e75a/users/651cc76d1fd14ce7bd28e9d51033f174/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.392333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3f161655e74064a3565a298e35e75a, user_id=651cc76d1fd14ce7bd28e9d51033f174, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:11.395116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:11.404399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:11.413076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1340/2654] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:11 2022] PUT /identity/v3/projects/9c3f161655e74064a3565a298e35e75a/users/651cc76d1fd14ce7bd28e9d51033f174/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:11.417094 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:11.428637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a904ba-f918-438d-9b8d-ec7eb7fcc998 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:11.503189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1874cf3-dfee-4072-ace5-a7dfbdef9e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:11.503987 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1874cf3-dfee-4072-ace5-a7dfbdef9e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.504345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1874cf3-dfee-4072-ace5-a7dfbdef9e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.504753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1874cf3-dfee-4072-ace5-a7dfbdef9e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.505731 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1874cf3-dfee-4072-ace5-a7dfbdef9e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:11.507722 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1874cf3-dfee-4072-ace5-a7dfbdef9e52 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:11.524333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1315/2655] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:11.525570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:11.536132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc0b18-bcc1-42f7-baea-05943ccccefb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:11.540840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:11.541774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.542053 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.542332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] PATH_INFO: `/v3/projects/9c3f161655e74064a3565a298e35e75a/users/651cc76d1fd14ce7bd28e9d51033f174/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.544476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3f161655e74064a3565a298e35e75a, user_id=651cc76d1fd14ce7bd28e9d51033f174, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:11.547459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:11.563610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1341/2656] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:11 2022] PUT /identity/v3/projects/9c3f161655e74064a3565a298e35e75a/users/651cc76d1fd14ce7bd28e9d51033f174/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:11.575638 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.575927 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.576210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.605648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] MFA Rules not processed for user `651cc76d1fd14ce7bd28e9d51033f174`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:11.606879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:11.698844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1316/2657] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:11 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:11.711799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:11.713676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:11.814808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f1196-14aa-4667-b7d7-7c5e40b4343b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:11.827963 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2d45b-0e38-425e-bc95-d77032d8d063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:11.828755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d45b-0e38-425e-bc95-d77032d8d063 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:11.829113 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d45b-0e38-425e-bc95-d77032d8d063 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:11.829439 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d45b-0e38-425e-bc95-d77032d8d063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:11.842739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2d45b-0e38-425e-bc95-d77032d8d063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:11.846062 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2d45b-0e38-425e-bc95-d77032d8d063 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:11.864755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1342/2658] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:11 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:12.076685 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:12.078442 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:12.094556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8a1d8-3b8d-4552-b8c8-4ac6cca78a48 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:12.098805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:12.104711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:12.104711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:12.104711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:12.170280 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:12.212095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:12.212436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:12.224279 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1317/2659] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:13.016209 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6371-5be6-447c-9414-7b6a24b7c69b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:13.017040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6371-5be6-447c-9414-7b6a24b7c69b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:13.017040 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6371-5be6-447c-9414-7b6a24b7c69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:13.036449 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-eeee6371-5be6-447c-9414-7b6a24b7c69b None None] MFA Rules not processed for user `4873db0f2fbf4a77a617f2102a5ccfae`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:13.037730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-eeee6371-5be6-447c-9414-7b6a24b7c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:13.104207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1343/2660] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:18.743775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:18.745251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:18.824677 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef0f7d-759b-4f14-85bd-d5d620c5b400 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:18.829718 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6657da-e23c-4b98-b005-358516eea309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:18.830564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6657da-e23c-4b98-b005-358516eea309 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:18.830854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6657da-e23c-4b98-b005-358516eea309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:18.831262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6657da-e23c-4b98-b005-358516eea309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:18.833055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5c6657da-e23c-4b98-b005-358516eea309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:18.926788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6657da-e23c-4b98-b005-358516eea309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:18.928441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6657da-e23c-4b98-b005-358516eea309 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:18.940324 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1318/2661] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:22.228680 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6371-5be6-447c-9414-7b6a24b7c69b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:22.316020 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6371-5be6-447c-9414-7b6a24b7c69b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:22.320698 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:22.321610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:22.321898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:22.322176 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] PATH_INFO: `/v3/users/8814041ab313448ca967886388ee53ae` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:22.323388 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8814041ab313448ca967886388ee53ae)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:22.325461 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:22.365257 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:22.380562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8814041ab313448ca967886388ee53ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:22.398613 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8814041ab313448ca967886388ee53ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:22.399455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8814041ab313448ca967886388ee53ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:22.416565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8814041ab313448ca967886388ee53ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:22.417297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8814041ab313448ca967886388ee53ae'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:22.433808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1344/2662] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:22 2022] DELETE /identity/v3/users/8814041ab313448ca967886388ee53ae => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:22.440790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:22.441386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:22.441386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:22.465831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:22.468265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:22.547489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1319/2663] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:22.558014 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:22.559513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:22.635146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef0a2b-b88d-4f87-9b9d-c0f8c3f9eb99 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:22.640030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:22.640752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:22.641130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:22.641562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:22.656701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:22.659436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:22.672438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1345/2664] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:22.905195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:22.907232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:22.920135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4761eb-256d-47b9-a166-4e7b2ead3a68 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:22.928332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:22.928332 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:22.928840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:22.929160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] PATH_INFO: `/v3/users/ec6639b5bf58466a945b080bd820dde9` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:22.930642 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6639b5bf58466a945b080bd820dde9)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:22.935148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:22.981301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:23.026961 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6639b5bf58466a945b080bd820dde9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.027833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6639b5bf58466a945b080bd820dde9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.041243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6639b5bf58466a945b080bd820dde9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.042003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6639b5bf58466a945b080bd820dde9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.057320 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-24869013-225d-427c-9734-9f4f734345ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6639b5bf58466a945b080bd820dde9'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.079252 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:23.080959 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:23.099696 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2388a4-fe0d-43d3-afeb-77f9f4468b0b service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:23.102831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1320/2665] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:22 2022] DELETE /identity/v3/users/ec6639b5bf58466a945b080bd820dde9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:23.124384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:23.125440 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:23.127845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:23.168419 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:23.170532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:23.207027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:23.208375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:23.208375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:23.208670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] PATH_INFO: `/v3/projects/7d92fab7a6c54f71a75db3f2e7236fc2` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:23.209826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d92fab7a6c54f71a75db3f2e7236fc2)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:23.211797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:23.254039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1321/2666] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:23.266711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:23.268448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:23.302000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d92fab7a6c54f71a75db3f2e7236fc2', 'request_id': 'req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.315841 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:23.318417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d92fab7a6c54f71a75db3f2e7236fc2', 'request_id': 'req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.335753 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d92fab7a6c54f71a75db3f2e7236fc2', 'request_id': 'req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.359450 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] The token cache is being invalidate because project 7d92fab7a6c54f71a75db3f2e7236fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:23.360170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.361429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.364505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1346/2667] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:23 2022] DELETE /identity/v3/projects/7d92fab7a6c54f71a75db3f2e7236fc2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:23.369873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:23.371008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bc766-f8d9-4529-bd08-81504370de6e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:23.371942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:23.488831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c868131-0ef0-42e4-be6b-8eac36f1ae0f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:23.493344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:23.495648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:23.496426 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:23.497207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:23.501936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:23.502430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:23.503989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:23.504671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:23.505332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] PATH_INFO: `/v3/users/651cc76d1fd14ce7bd28e9d51033f174` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:23.508341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=651cc76d1fd14ce7bd28e9d51033f174)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:23.512799 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:23.521450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:23.525236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:23.533258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:23.542449 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1322/2668] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:23.598445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651cc76d1fd14ce7bd28e9d51033f174'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.619050 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651cc76d1fd14ce7bd28e9d51033f174'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.620781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651cc76d1fd14ce7bd28e9d51033f174'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.636500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651cc76d1fd14ce7bd28e9d51033f174'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.637958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651cc76d1fd14ce7bd28e9d51033f174'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:23.685845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1347/2669] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:23 2022] DELETE /identity/v3/users/651cc76d1fd14ce7bd28e9d51033f174 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:24.416762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:24.418368 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:24.445355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867f0b6-b749-4ab8-979b-6c166145d189 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:24.543986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:24.546878 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:24.558008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:24.558915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:24.559214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:24.559497 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] PATH_INFO: `/v3/projects/a2dc1a0a6ac041449241778f044e21ce` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:24.561441 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2dc1a0a6ac041449241778f044e21ce)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:24.563704 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:24.662003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2dc1a0a6ac041449241778f044e21ce', 'request_id': 'req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:24.674182 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2dc1a0a6ac041449241778f044e21ce', 'request_id': 'req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:24.683568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2dc1a0a6ac041449241778f044e21ce', 'request_id': 'req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:24.685990 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796565b-a9fc-435f-a85d-3e9685a1bd7a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:24.701610 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:24.704079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] The token cache is being invalidate because project a2dc1a0a6ac041449241778f044e21ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:24.704439 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:24.707120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f4dddfa7-34a0-4b99-86ca-a4e5d61e7a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:24.710220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1323/2670] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:24 2022] DELETE /identity/v3/projects/a2dc1a0a6ac041449241778f044e21ce => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:24.816196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:24.818006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:24.818432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:24.818905 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] PATH_INFO: `/v3/projects/9c3f161655e74064a3565a298e35e75a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:24.820916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3f161655e74064a3565a298e35e75a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:24.825157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:24.934375 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3f161655e74064a3565a298e35e75a', 'request_id': 'req-f61b5769-db32-4edf-866c-9ba1e355aef3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:24.947330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3f161655e74064a3565a298e35e75a', 'request_id': 'req-f61b5769-db32-4edf-866c-9ba1e355aef3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:24.987092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3f161655e74064a3565a298e35e75a', 'request_id': 'req-f61b5769-db32-4edf-866c-9ba1e355aef3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:25.044373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] The token cache is being invalidate because project 9c3f161655e74064a3565a298e35e75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:25.045406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:25.046310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-f61b5769-db32-4edf-866c-9ba1e355aef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:25.048429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1348/2671] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:24 2022] DELETE /identity/v3/projects/9c3f161655e74064a3565a298e35e75a => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:25.055570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1621e-80bd-40ab-a67f-a3f33cb85742 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.057634 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1621e-80bd-40ab-a67f-a3f33cb85742 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.057962 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1621e-80bd-40ab-a67f-a3f33cb85742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.097374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9cb1621e-80bd-40ab-a67f-a3f33cb85742 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:25.098598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9cb1621e-80bd-40ab-a67f-a3f33cb85742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:25.240080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1324/2672] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:25.248316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4196-1f6c-420a-807a-04a39be6f77e None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.248606 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4196-1f6c-420a-807a-04a39be6f77e None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.249337 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4196-1f6c-420a-807a-04a39be6f77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.269213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-604a4196-1f6c-420a-807a-04a39be6f77e None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:25.270494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-604a4196-1f6c-420a-807a-04a39be6f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:25.278625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1349/2673] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:25.283559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1621e-80bd-40ab-a67f-a3f33cb85742 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.295344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1621e-80bd-40ab-a67f-a3f33cb85742 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.301208 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.301886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.302229 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.302453 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.303244 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:11:25.303500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:25.305177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:25.318817 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1325/2674] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:11:25.323103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a4196-1f6c-420a-807a-04a39be6f77e None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.333359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a4196-1f6c-420a-807a-04a39be6f77e None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.338146 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.338825 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.339116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.339417 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.340128 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:25.342652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:25.370811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1350/2675] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:25 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:25.375301 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.387095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f76e9-5de0-4a0e-999d-3645bb84e9c5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.403049 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:25.464490 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.465429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.465856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.466212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.467562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:25.469553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:25.494135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1326/2676] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:25 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:25.498906 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.511355 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d87b1-b75d-40f7-8176-04864592c546 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.516855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.517621 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.517911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.518193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.518940 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:25.520574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:25.533414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1351/2677] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:11:25.538402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.549630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67909c-a92a-4085-94ef-653a1ceb4f6d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.556598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73b00d-eeb4-4f7f-868a-9a2e7ba5fedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.557325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73b00d-eeb4-4f7f-868a-9a2e7ba5fedc admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.557618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73b00d-eeb4-4f7f-868a-9a2e7ba5fedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.557905 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73b00d-eeb4-4f7f-868a-9a2e7ba5fedc admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.559405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73b00d-eeb4-4f7f-868a-9a2e7ba5fedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:25.560539 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73b00d-eeb4-4f7f-868a-9a2e7ba5fedc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:25.583037 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1327/2678] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:25.587064 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.601422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fbd72-8fe5-4bf1-8ef6-8fdda66d6b4a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.607789 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.608711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.609039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.609300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] PATH_INFO: `/v3/projects/591e8108f6e94e709200f01b1ad31f12/users/990538c2fc8b4e09befde16f19c76082/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.629049 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=591e8108f6e94e709200f01b1ad31f12, user_id=990538c2fc8b4e09befde16f19c76082, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:25.632089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:25.653018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1352/2679] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:25 2022] PUT /identity/v3/projects/591e8108f6e94e709200f01b1ad31f12/users/990538c2fc8b4e09befde16f19c76082/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:25.664321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fe76-28d1-4c7b-9584-d66a1c94247d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.664548 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fe76-28d1-4c7b-9584-d66a1c94247d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.665127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fe76-28d1-4c7b-9584-d66a1c94247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.711770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-59b0fe76-28d1-4c7b-9584-d66a1c94247d None None] MFA Rules not processed for user `990538c2fc8b4e09befde16f19c76082`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:25.713097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-59b0fe76-28d1-4c7b-9584-d66a1c94247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:25.790734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1328/2680] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:25.796279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.844662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:25.869320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed08e6e-12dd-4e0d-9e58-d824bfdc8f4f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.874144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.874817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.875106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.875387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:25.876080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:25.878036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:25.906704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1353/2681] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:25 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:25.910991 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0fe76-28d1-4c7b-9584-d66a1c94247d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:25.925656 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0fe76-28d1-4c7b-9584-d66a1c94247d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:25.998697 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:25.999504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:25.999715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:25.999999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.000721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:26.002743 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:26.025835 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1329/2682] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:25 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:26.030862 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:26.045312 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95bfb-e4e2-4481-9a4a-2a8b8aad429f admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:26.053561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:26.054720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:26.055206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:26.055691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.057085 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:26.060258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:26.086986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1354/2683] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:11:26.093078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:26.110789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17039c33-d787-4627-97fa-cdd4ff6282cc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:26.116732 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:26.117701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:26.117988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:26.118269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] PATH_INFO: `/v3/projects/9c749d1d9f4a43b98e407f1950db251b/users/04d45e691e994c03bc7b1004361db21a/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.133594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:26.138217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c749d1d9f4a43b98e407f1950db251b, user_id=04d45e691e994c03bc7b1004361db21a, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:26.142428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:26.164239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1330/2684] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:26 2022] PUT /identity/v3/projects/9c749d1d9f4a43b98e407f1950db251b/users/04d45e691e994c03bc7b1004361db21a/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:26.168538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:26.183558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d147c-8c25-4bd9-ae2e-cf9d30db2a28 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:26.299978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0bf2e-4209-4246-8cbe-47a7c33b352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:26.301320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0bf2e-4209-4246-8cbe-47a7c33b352b admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:26.301892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0bf2e-4209-4246-8cbe-47a7c33b352b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:26.302378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0bf2e-4209-4246-8cbe-47a7c33b352b admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.303788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0bf2e-4209-4246-8cbe-47a7c33b352b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:26.307006 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0bf2e-4209-4246-8cbe-47a7c33b352b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:26.324226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1355/2685] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:11:26.329475 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:26.340969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698f44b-6555-449b-9a56-f1da3743f830 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:26.346231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:26.347198 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:26.347488 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:26.347775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] PATH_INFO: `/v3/projects/9c749d1d9f4a43b98e407f1950db251b/users/04d45e691e994c03bc7b1004361db21a/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.350236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c749d1d9f4a43b98e407f1950db251b, user_id=04d45e691e994c03bc7b1004361db21a, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:26.353522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:26.367316 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1331/2686] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:26 2022] PUT /identity/v3/projects/9c749d1d9f4a43b98e407f1950db251b/users/04d45e691e994c03bc7b1004361db21a/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:26.379180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:26.379735 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:26.380032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.414464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] MFA Rules not processed for user `04d45e691e994c03bc7b1004361db21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:26.416776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:26.509631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:26.515456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:26.570487 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1356/2687] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:11:26.582350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:26.583733 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:26.589917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c888a7-2685-49a2-855b-af206f3a3163 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:26.594613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:26.595296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:26.595590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:26.595874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.597777 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:26.623666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:26.650080 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727859b-bbb8-498a-92c8-2474177750b2 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:26.656131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b92a2-3f4f-4983-8a79-a03faaace637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:26.657500 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b92a2-3f4f-4983-8a79-a03faaace637 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:26.657993 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b92a2-3f4f-4983-8a79-a03faaace637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:26.658524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b92a2-3f4f-4983-8a79-a03faaace637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:26.670606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:26.672328 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:26.674926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b92a2-3f4f-4983-8a79-a03faaace637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:26.677972 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b92a2-3f4f-4983-8a79-a03faaace637 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:26.684107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1332/2688] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:26.698761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1357/2689] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:11:27.158112 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:27.172899 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5aef4-aff9-4d38-aaab-fb9172b81cf3 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:27.181604 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:27.182620 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:27.182908 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:27.183289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] PATH_INFO: `/v3/users/990538c2fc8b4e09befde16f19c76082` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:27.184633 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=990538c2fc8b4e09befde16f19c76082)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:27.187755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:27.243752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '990538c2fc8b4e09befde16f19c76082'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:27.244690 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '990538c2fc8b4e09befde16f19c76082'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:27.253029 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '990538c2fc8b4e09befde16f19c76082'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:27.253774 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '990538c2fc8b4e09befde16f19c76082'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:27.274937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '990538c2fc8b4e09befde16f19c76082'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:27.287723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:27.306761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1333/2690] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:27 2022] DELETE /identity/v3/users/990538c2fc8b4e09befde16f19c76082 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:27.311593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:27.311955 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:27.312299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:27.333588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:27.335039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:27.422612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1358/2691] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:27.437883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:27.439417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:27.520828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c24d-726c-499d-b883-419ee9149edb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:27.525103 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:27.526034 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:27.526564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:27.526807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:27.540118 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:27.541715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:27.554994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1334/2692] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:27.888818 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:27.893806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:27.920946 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b1f57-8337-4edc-bf23-b9022025c749 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:27.929743 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:27.931260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:27.931794 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:27.932290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] PATH_INFO: `/v3/projects/591e8108f6e94e709200f01b1ad31f12` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:27.934358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591e8108f6e94e709200f01b1ad31f12)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:27.938205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:28.018595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591e8108f6e94e709200f01b1ad31f12', 'request_id': 'req-e8968e5b-03f0-4b07-b542-321da0db78b5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.028039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591e8108f6e94e709200f01b1ad31f12', 'request_id': 'req-e8968e5b-03f0-4b07-b542-321da0db78b5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.036549 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591e8108f6e94e709200f01b1ad31f12', 'request_id': 'req-e8968e5b-03f0-4b07-b542-321da0db78b5'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.052632 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] The token cache is being invalidate because project 591e8108f6e94e709200f01b1ad31f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:28.052934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.053704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.059819 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1359/2693] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:27 2022] DELETE /identity/v3/projects/591e8108f6e94e709200f01b1ad31f12 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:28.062161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:28.077043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d79e33-794f-403a-a354-b5c9474cd1b6 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:28.149673 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:28.154217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:28.154217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:28.154217 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] PATH_INFO: `/v3/users/04d45e691e994c03bc7b1004361db21a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:28.155930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d45e691e994c03bc7b1004361db21a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:28.158170 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:28.226971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d45e691e994c03bc7b1004361db21a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.227920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d45e691e994c03bc7b1004361db21a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.236514 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d45e691e994c03bc7b1004361db21a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.237428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d45e691e994c03bc7b1004361db21a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.255783 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3370108f-8794-4eff-b220-7aeddfb1ee8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d45e691e994c03bc7b1004361db21a'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:28.273151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1335/2694] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:28 2022] DELETE /identity/v3/users/04d45e691e994c03bc7b1004361db21a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:28.815722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:28.817591 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:28.864847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968e5b-03f0-4b07-b542-321da0db78b5 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:28.946474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:28.947562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:28.947866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:28.948130 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] PATH_INFO: `/v3/projects/9c749d1d9f4a43b98e407f1950db251b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:28.953983 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c749d1d9f4a43b98e407f1950db251b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:28.956617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:28.958467 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:29.028198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c749d1d9f4a43b98e407f1950db251b', 'request_id': 'req-39118eec-8104-4db9-afd8-989d98b18699'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:29.042045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c749d1d9f4a43b98e407f1950db251b', 'request_id': 'req-39118eec-8104-4db9-afd8-989d98b18699'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:29.054686 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c749d1d9f4a43b98e407f1950db251b', 'request_id': 'req-39118eec-8104-4db9-afd8-989d98b18699'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:29.078403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] The token cache is being invalidate because project 9c749d1d9f4a43b98e407f1950db251b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:29.079286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:29.080197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-39118eec-8104-4db9-afd8-989d98b18699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:29.083588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1360/2695] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:28 2022] DELETE /identity/v3/projects/9c749d1d9f4a43b98e407f1950db251b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:29.091083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6e8f2-dabc-4b42-b7c7-bfa1b505bf25 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.092350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6e8f2-dabc-4b42-b7c7-bfa1b505bf25 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.092911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6e8f2-dabc-4b42-b7c7-bfa1b505bf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.126150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e2e6e8f2-dabc-4b42-b7c7-bfa1b505bf25 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:29.127880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e2e6e8f2-dabc-4b42-b7c7-bfa1b505bf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:29.155940 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:29.225199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1336/2696] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:29.234485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7472b-ddf9-4ff5-a2be-72eb8fae522c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.234958 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7472b-ddf9-4ff5-a2be-72eb8fae522c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.235289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7472b-ddf9-4ff5-a2be-72eb8fae522c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.270944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ebe7472b-ddf9-4ff5-a2be-72eb8fae522c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:29.274365 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ebe7472b-ddf9-4ff5-a2be-72eb8fae522c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:29.292595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6e8f2-dabc-4b42-b7c7-bfa1b505bf25 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:29.293275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1361/2697] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:29.305218 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6e8f2-dabc-4b42-b7c7-bfa1b505bf25 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:29.316792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:29.318032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.318715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.318715 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.320450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:11:29.320721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:29.326418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:29.351379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1337/2698] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:11:29.352851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7472b-ddf9-4ff5-a2be-72eb8fae522c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:29.364523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7472b-ddf9-4ff5-a2be-72eb8fae522c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:29.371267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:29.372126 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.372519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.373174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.374491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:29.376587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:29.422471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1362/2699] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:29 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:29.426822 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:29.440171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a95948-ac58-435f-8309-97dd194664f5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:29.550055 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:29.551344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.551828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.552278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.553253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:29.555720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:29.586755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1338/2700] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:29 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:29.588089 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:29.603982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248d36a-abf7-43ec-8dde-a26298e958c2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:29.610831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:29.611519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.611800 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.612074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.612829 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:29.614534 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:29.629640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1363/2701] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:29.631573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:29.644920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041decc-e826-42d3-b0ed-df683a69a37a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:29.650898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e25fb3-39ef-4450-937e-885fb20197ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:29.651173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e25fb3-39ef-4450-937e-885fb20197ba admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.651543 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e25fb3-39ef-4450-937e-885fb20197ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.651746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e25fb3-39ef-4450-937e-885fb20197ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.652384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e25fb3-39ef-4450-937e-885fb20197ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:29.655667 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e25fb3-39ef-4450-937e-885fb20197ba admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:29.669122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1339/2702] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:29.673695 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:29.693070 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ed50b-2f32-426f-bd5e-295e0f0808e5 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:29.703022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:29.704320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.704608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.705100 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] PATH_INFO: `/v3/projects/31271efeb0e549b29db0b3ebba6208f0/users/ccffb7f17de44ef5bcea2a403f5f487d/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.724483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31271efeb0e549b29db0b3ebba6208f0, user_id=ccffb7f17de44ef5bcea2a403f5f487d, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:29.727194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:29.752813 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1364/2703] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:29 2022] PUT /identity/v3/projects/31271efeb0e549b29db0b3ebba6208f0/users/ccffb7f17de44ef5bcea2a403f5f487d/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:29.856625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a050f-4114-4920-a3e8-ecc46179efa7 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:29.857291 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a050f-4114-4920-a3e8-ecc46179efa7 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:29.857816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a050f-4114-4920-a3e8-ecc46179efa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:29.923325 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-885a050f-4114-4920-a3e8-ecc46179efa7 None None] MFA Rules not processed for user `ccffb7f17de44ef5bcea2a403f5f487d`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:29.926343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-885a050f-4114-4920-a3e8-ecc46179efa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:30.072379 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1340/2704] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:30.078258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:30.157107 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c3e29-afc1-42b4-a268-83589ca28447 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:30.162803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:30.163435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:30.163726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:30.164021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:30.164776 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:30.166765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:30.232672 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1365/2705] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:30 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:11:30.233917 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a050f-4114-4920-a3e8-ecc46179efa7 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:30.249175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a050f-4114-4920-a3e8-ecc46179efa7 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:30.276308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:30.363877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:30.364717 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:30.365057 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:30.365346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:30.366148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:30.368338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:30.400086 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1341/2706] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:30 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:11:30.406537 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:30.426927 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a35c6-3b11-468c-bd13-fbd61edc45ea admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:30.435995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:30.437360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:30.437954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:30.438548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:30.440090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:30.443513 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:30.458316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:30.483088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:30.488689 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1366/2707] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:30.524327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c635e-de20-4483-85d7-df42af9694c3 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:30.536189 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:30.538793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:30.539166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:30.539517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] PATH_INFO: `/v3/projects/8374c80ab3af4e0e828c830c58f1f652/users/213df2f75e534abd84ef58539b42c31b/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:30.563121 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8374c80ab3af4e0e828c830c58f1f652, user_id=213df2f75e534abd84ef58539b42c31b, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:30.571517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:30.616958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1342/2708] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:30 2022] PUT /identity/v3/projects/8374c80ab3af4e0e828c830c58f1f652/users/213df2f75e534abd84ef58539b42c31b/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:30.621679 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:30.645508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa670c-cc09-4806-b9bb-4d11c032e486 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:30.787710 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341bfc7-9341-4e07-b764-13325efd008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:30.788406 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341bfc7-9341-4e07-b764-13325efd008c admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:30.788730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341bfc7-9341-4e07-b764-13325efd008c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:30.789038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341bfc7-9341-4e07-b764-13325efd008c admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:30.789762 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341bfc7-9341-4e07-b764-13325efd008c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:30.791427 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341bfc7-9341-4e07-b764-13325efd008c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:30.806230 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1367/2709] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:30.812790 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:30.839771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6a27-ae18-495d-b036-11710a1c13bc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:30.855370 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:30.856413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:30.858450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:30.858836 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] PATH_INFO: `/v3/projects/8374c80ab3af4e0e828c830c58f1f652/users/213df2f75e534abd84ef58539b42c31b/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:30.864486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8374c80ab3af4e0e828c830c58f1f652, user_id=213df2f75e534abd84ef58539b42c31b, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:30.867755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:30.896108 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1343/2710] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:30 2022] PUT /identity/v3/projects/8374c80ab3af4e0e828c830c58f1f652/users/213df2f75e534abd84ef58539b42c31b/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:30.916116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bcfed-289f-4d7b-aa32-e2b4827c5891 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:30.916752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bcfed-289f-4d7b-aa32-e2b4827c5891 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:30.917127 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bcfed-289f-4d7b-aa32-e2b4827c5891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:30.984335 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bf9bcfed-289f-4d7b-aa32-e2b4827c5891 None None] MFA Rules not processed for user `213df2f75e534abd84ef58539b42c31b`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:30.985755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bf9bcfed-289f-4d7b-aa32-e2b4827c5891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:31.147471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1368/2711] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:31.169922 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:31.173600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:31.312579 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6fb6-322d-4110-9a4f-ae331047e47a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:31.317436 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:31.318132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:31.318423 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:31.320012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:31.331139 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:31.333971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:31.350913 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1344/2712] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:31.528526 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bcfed-289f-4d7b-aa32-e2b4827c5891 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:31.545366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bcfed-289f-4d7b-aa32-e2b4827c5891 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:31.551120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa0d6d-c1cf-447f-9f17-33508e0313d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:31.551806 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa0d6d-c1cf-447f-9f17-33508e0313d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:31.552093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa0d6d-c1cf-447f-9f17-33508e0313d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:31.552373 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa0d6d-c1cf-447f-9f17-33508e0313d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:31.649494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa0d6d-c1cf-447f-9f17-33508e0313d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:31.652674 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa0d6d-c1cf-447f-9f17-33508e0313d0 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:31.677412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1369/2713] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:32.106546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:32.225174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523c081-8abb-4640-9d2d-8f6c8d0138da service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:32.234549 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:32.236504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:32.237222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:32.237831 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] PATH_INFO: `/v3/users/ccffb7f17de44ef5bcea2a403f5f487d` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:32.240493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccffb7f17de44ef5bcea2a403f5f487d)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:32.244754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:32.268946 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:32.338967 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccffb7f17de44ef5bcea2a403f5f487d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:32.340102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccffb7f17de44ef5bcea2a403f5f487d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:32.356009 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccffb7f17de44ef5bcea2a403f5f487d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:32.357911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccffb7f17de44ef5bcea2a403f5f487d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:32.378486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccffb7f17de44ef5bcea2a403f5f487d'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:32.395904 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1345/2714] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:32 2022] DELETE /identity/v3/users/ccffb7f17de44ef5bcea2a403f5f487d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:32.403034 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:32.403588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:32.404188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:32.434925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:32.437055 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:32.461932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=100 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:32.564453 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1370/2715] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:32.577975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:32.580714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:32.681906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c4ba-2bd3-47bb-9aff-f0607bba88b6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:32.688214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:32.689303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:32.689486 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:32.689785 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:32.711143 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:32.714203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:32.731476 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1346/2716] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:33.080588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:33.082267 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:33.106077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8e51a-a1ba-4f7e-925d-a16420f1dfb5 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:33.112857 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:33.113811 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:33.114132 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:33.114379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] PATH_INFO: `/v3/projects/31271efeb0e549b29db0b3ebba6208f0` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:33.115869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] RBAC: Authorizing `identity:delete_project(project_id=31271efeb0e549b29db0b3ebba6208f0)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:33.117923 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:33.197215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31271efeb0e549b29db0b3ebba6208f0', 'request_id': 'req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.207093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31271efeb0e549b29db0b3ebba6208f0', 'request_id': 'req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.220024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31271efeb0e549b29db0b3ebba6208f0', 'request_id': 'req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.246217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] The token cache is being invalidate because project 31271efeb0e549b29db0b3ebba6208f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:33.246745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.247978 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.251359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1371/2717] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:33 2022] DELETE /identity/v3/projects/31271efeb0e549b29db0b3ebba6208f0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:33.256848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:33.268931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d585-4531-49da-9ee8-0cd4f21cceb5 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:33.353462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:33.358781 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:33.360776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:33.361505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] PATH_INFO: `/v3/users/213df2f75e534abd84ef58539b42c31b` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:33.364749 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213df2f75e534abd84ef58539b42c31b)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:33.368432 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:33.439102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:33.449181 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213df2f75e534abd84ef58539b42c31b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.450007 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213df2f75e534abd84ef58539b42c31b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.458951 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213df2f75e534abd84ef58539b42c31b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.459752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213df2f75e534abd84ef58539b42c31b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.474196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3dea6c1b-337b-454f-958e-605f4aa09df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213df2f75e534abd84ef58539b42c31b'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:33.495344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1347/2718] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:33 2022] DELETE /identity/v3/users/213df2f75e534abd84ef58539b42c31b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:33.884460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:33.887147 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:33.927167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10f81-ced2-43a8-b4fb-f2424e29a13c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:34.048756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:34.050595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.051097 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.051580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] PATH_INFO: `/v3/projects/8374c80ab3af4e0e828c830c58f1f652` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.054150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8374c80ab3af4e0e828c830c58f1f652)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:34.058205 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:34.137522 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:34.142771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8374c80ab3af4e0e828c830c58f1f652', 'request_id': 'req-3c31ee82-503d-4e82-a8fa-069f84c44bf3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:34.153386 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8374c80ab3af4e0e828c830c58f1f652', 'request_id': 'req-3c31ee82-503d-4e82-a8fa-069f84c44bf3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:34.172144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8374c80ab3af4e0e828c830c58f1f652', 'request_id': 'req-3c31ee82-503d-4e82-a8fa-069f84c44bf3'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:34.198904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] The token cache is being invalidate because project 8374c80ab3af4e0e828c830c58f1f652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:34.199283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:34.200360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-3c31ee82-503d-4e82-a8fa-069f84c44bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:34.203121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1372/2719] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:33 2022] DELETE /identity/v3/projects/8374c80ab3af4e0e828c830c58f1f652 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:34.210862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78172eb-dc5d-41b8-adc6-5573fe992014 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.212260 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78172eb-dc5d-41b8-adc6-5573fe992014 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.212568 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78172eb-dc5d-41b8-adc6-5573fe992014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.233083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-a78172eb-dc5d-41b8-adc6-5573fe992014 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:34.234355 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a78172eb-dc5d-41b8-adc6-5573fe992014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:34.339760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1348/2720] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:34.350642 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e649c-10fb-4a2a-afa9-ce3f8e614a19 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.351226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e649c-10fb-4a2a-afa9-ce3f8e614a19 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.351724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e649c-10fb-4a2a-afa9-ce3f8e614a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.386498 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-768e649c-10fb-4a2a-afa9-ce3f8e614a19 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:34.388607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-768e649c-10fb-4a2a-afa9-ce3f8e614a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:34.402748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1373/2721] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:34.409674 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78172eb-dc5d-41b8-adc6-5573fe992014 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:34.423016 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78172eb-dc5d-41b8-adc6-5573fe992014 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:34.430806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:34.431776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.432061 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.432346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.433133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:11:34.433438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:34.435816 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:34.451413 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1349/2722] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:11:34.457274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e649c-10fb-4a2a-afa9-ce3f8e614a19 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:34.474066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e649c-10fb-4a2a-afa9-ce3f8e614a19 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:34.486962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:34.488460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.489002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.489633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.492425 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:34.505991 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:34.549598 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1374/2723] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:34 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:34.551688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:34.562850 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde583a9-9981-4a31-a6ea-6d8fed672476 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:34.649861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:34.651059 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.651602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.652048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.653203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:34.655882 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:34.682283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1350/2724] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:34 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:11:34.687334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:34.698304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18848-0d59-4cf6-8498-c3114fdba700 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:34.703894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:34.704581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.704869 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.705185 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.705934 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:34.707574 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:34.723228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1375/2725] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:11:34.724532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:34.735936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22120d-342b-4a2b-a186-829d229233d0 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:34.740655 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0271b-f819-457d-a6c5-981d92d53b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:34.741366 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0271b-f819-457d-a6c5-981d92d53b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.741652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0271b-f819-457d-a6c5-981d92d53b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.741925 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0271b-f819-457d-a6c5-981d92d53b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.742669 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0271b-f819-457d-a6c5-981d92d53b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:34.744381 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0271b-f819-457d-a6c5-981d92d53b20 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:34.759860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1351/2726] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:34.761105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:34.772412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60d475-534a-4622-94d1-6121a7ba68e2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:34.778196 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:34.779781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.780144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.780748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] PATH_INFO: `/v3/projects/9bfdb972b07c4f90ba7dd94b122cd092/users/b4e8a51564c04af5888e02975579c5b6/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.804021 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bfdb972b07c4f90ba7dd94b122cd092, user_id=b4e8a51564c04af5888e02975579c5b6, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:34.807304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:34.832240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1376/2727] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:34 2022] PUT /identity/v3/projects/9bfdb972b07c4f90ba7dd94b122cd092/users/b4e8a51564c04af5888e02975579c5b6/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:34.859792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8bbca-57b3-4d67-ba14-d3d698c6dfb2 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:34.860087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8bbca-57b3-4d67-ba14-d3d698c6dfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:34.860380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8bbca-57b3-4d67-ba14-d3d698c6dfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:34.909348 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-29d8bbca-57b3-4d67-ba14-d3d698c6dfb2 None None] MFA Rules not processed for user `b4e8a51564c04af5888e02975579c5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:34.910980 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-29d8bbca-57b3-4d67-ba14-d3d698c6dfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:34.923481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:35.020939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1352/2728] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:35.025739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.115115 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcc745-e2b0-4a62-a23c-4287df2b25ee admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.120962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.121697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.121979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.122935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.123663 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.125633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.165419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1377/2729] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:35 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:11:35.171807 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8bbca-57b3-4d67-ba14-d3d698c6dfb2 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.189166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8bbca-57b3-4d67-ba14-d3d698c6dfb2 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.289450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.290107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.290496 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.290792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.292003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.299205 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.326796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1353/2730] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:35 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:35.331269 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.345088 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748220bd-8bbc-4eaa-b49f-bbcecbe5fecf admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.350628 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.351459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.351864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.352218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.353106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.354903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.373227 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1378/2731] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:35.376856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.389644 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f07e07-fc79-476d-96c5-da56f51a94bd admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.399837 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b902342-6f98-4e00-86b1-707a51d3b248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.401266 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902342-6f98-4e00-86b1-707a51d3b248 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.401613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902342-6f98-4e00-86b1-707a51d3b248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.402096 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902342-6f98-4e00-86b1-707a51d3b248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.402883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b902342-6f98-4e00-86b1-707a51d3b248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.404750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b902342-6f98-4e00-86b1-707a51d3b248 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.418447 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1354/2732] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:35.423197 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.433881 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4f2bf-51ca-43c2-9a12-90d90bab7d28 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.439344 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.440247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.440531 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.440810 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] PATH_INFO: `/v3/projects/48bd5e9ccf8146db81631f00a1b97f0a/users/8f13412e2b9e445b97c4985cde653a70/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.461559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bd5e9ccf8146db81631f00a1b97f0a, user_id=8f13412e2b9e445b97c4985cde653a70, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.464326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.497360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1379/2733] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:35 2022] PUT /identity/v3/projects/48bd5e9ccf8146db81631f00a1b97f0a/users/8f13412e2b9e445b97c4985cde653a70/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:35.499823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7267d08-2d8b-462f-98a7-f4bf9ab7372f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.500179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7267d08-2d8b-462f-98a7-f4bf9ab7372f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.500391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7267d08-2d8b-462f-98a7-f4bf9ab7372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.555296 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e7267d08-2d8b-462f-98a7-f4bf9ab7372f None None] MFA Rules not processed for user `8f13412e2b9e445b97c4985cde653a70`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:35.556564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e7267d08-2d8b-462f-98a7-f4bf9ab7372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:35.651916 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1355/2734] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:11:35.657308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.697263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:35.773241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b724-fbdd-46bf-be02-d0cc7804bcc0 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.779323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.779763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.780046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.780320 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.781084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.783036 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.825177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1380/2735] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:11:35 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:11:35.826521 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7267d08-2d8b-462f-98a7-f4bf9ab7372f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.838939 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7267d08-2d8b-462f-98a7-f4bf9ab7372f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.852828 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:35.913597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.914369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.914725 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.915076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.915860 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.917975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.928235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.932258 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:35.945928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1356/2736] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:11:35 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:11:35.949495 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.952654 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2841710-14ce-498f-8240-f6334eba91d6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.960388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3ee4f-a4f7-4073-bbe5-0b858dac9a54 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:35.968004 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.968820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.969120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.969408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.970152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.971228 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:35.972041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.973189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:35.973701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:35.974334 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] PATH_INFO: `/v3/users/de6636a0522642799e43a8f02ee602ea` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:35.976892 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de6636a0522642799e43a8f02ee602ea)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:35.980951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:35.986575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1357/2737] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:11:35.987622 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:35.998327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c9a4-d962-4dd3-8d25-3db631858025 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:36.003071 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:36.003959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:36.004243 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:36.004518 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] PATH_INFO: `/v3/projects/510dd0a5703149cc804a327525e9c4b3/users/b2c5c4f811c049e9a210d7fcaf2eca58/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:36.024104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] RBAC: Authorizing `identity:create_grant(project_id=510dd0a5703149cc804a327525e9c4b3, user_id=b2c5c4f811c049e9a210d7fcaf2eca58, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:36.026883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:36.039767 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de6636a0522642799e43a8f02ee602ea'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.044492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1358/2738] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:35 2022] PUT /identity/v3/projects/510dd0a5703149cc804a327525e9c4b3/users/b2c5c4f811c049e9a210d7fcaf2eca58/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:36.048532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:36.054774 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de6636a0522642799e43a8f02ee602ea'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.055570 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de6636a0522642799e43a8f02ee602ea'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.066820 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de6636a0522642799e43a8f02ee602ea'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.068124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de6636a0522642799e43a8f02ee602ea'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.085394 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1381/2739] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:35 2022] DELETE /identity/v3/users/de6636a0522642799e43a8f02ee602ea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:36.128754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52955e34-9513-4692-a2b7-a28ac44d679e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:36.196832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd49cf6-4ce3-438f-8715-20e56e3f02ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:36.197562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd49cf6-4ce3-438f-8715-20e56e3f02ca admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:36.197851 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd49cf6-4ce3-438f-8715-20e56e3f02ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:36.198130 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd49cf6-4ce3-438f-8715-20e56e3f02ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:36.198885 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd49cf6-4ce3-438f-8715-20e56e3f02ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:36.201079 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd49cf6-4ce3-438f-8715-20e56e3f02ca admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:36.212294 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1359/2740] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:11:36.217636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:36.219347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:36.234098 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494ac01-28f9-4ccc-84c4-5266e42cbcc2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:36.239625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:36.240546 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:36.240826 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:36.241141 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] PATH_INFO: `/v3/projects/510dd0a5703149cc804a327525e9c4b3/users/b2c5c4f811c049e9a210d7fcaf2eca58/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:36.243330 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510dd0a5703149cc804a327525e9c4b3, user_id=b2c5c4f811c049e9a210d7fcaf2eca58, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:36.246159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:36.259667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1382/2741] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:11:36 2022] PUT /identity/v3/projects/510dd0a5703149cc804a327525e9c4b3/users/b2c5c4f811c049e9a210d7fcaf2eca58/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:36.272052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:36.272352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:36.272613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:36.312840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] MFA Rules not processed for user `b2c5c4f811c049e9a210d7fcaf2eca58`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:36.315080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:36.323873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:36.354005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:36.357817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:36.449123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1360/2742] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:36.468248 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:36.470151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:36.470837 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12618b-f6af-4795-97ad-b5a8dbee02e2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:36.476027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:36.476903 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:36.477436 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:36.477711 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] PATH_INFO: `/v3/projects/26a20f6f9f254aae91c952ee44a4ff74` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:36.478999 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a20f6f9f254aae91c952ee44a4ff74)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:36.480942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:36.489501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:36.553019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a20f6f9f254aae91c952ee44a4ff74', 'request_id': 'req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.561181 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a20f6f9f254aae91c952ee44a4ff74', 'request_id': 'req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.571154 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a20f6f9f254aae91c952ee44a4ff74', 'request_id': 'req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.572501 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e515d-8260-454f-92c9-8c7ff4cff7d4 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:36.588361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] The token cache is being invalidate because project 26a20f6f9f254aae91c952ee44a4ff74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:36.589275 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.590153 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.593424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1383/2743] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:36 2022] DELETE /identity/v3/projects/26a20f6f9f254aae91c952ee44a4ff74 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:36.598316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:36.601691 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:36.660142 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:36.660909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:36.661251 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:36.661533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:36.664950 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:36.695039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833be96-fd4c-4ecc-bf0c-f57e62f9f49e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:36.702548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:36.704016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:36.704514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:36.705029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] PATH_INFO: `/v3/users/c1d7d84cf594405aa0e7f5307fa60657` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:36.707277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d7d84cf594405aa0e7f5307fa60657)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:36.710428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:36.737035 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:36.738527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:36.751109 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1361/2744] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:11:36.793653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d7d84cf594405aa0e7f5307fa60657'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.817769 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d7d84cf594405aa0e7f5307fa60657'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.818877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d7d84cf594405aa0e7f5307fa60657'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.828225 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d7d84cf594405aa0e7f5307fa60657'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.828897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d7d84cf594405aa0e7f5307fa60657'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:36.848403 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1384/2745] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:36 2022] DELETE /identity/v3/users/c1d7d84cf594405aa0e7f5307fa60657 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:37.064365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:37.079226 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beed1e3-b535-4c75-a680-ba3b5a4b8908 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:37.090947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:37.152960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:37.155122 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:37.155388 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:37.155672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] PATH_INFO: `/v3/projects/ce40207353eb449c99cb7abc1ddc878f` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:37.157211 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce40207353eb449c99cb7abc1ddc878f)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:37.159100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:37.214986 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce40207353eb449c99cb7abc1ddc878f', 'request_id': 'req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:37.222456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce40207353eb449c99cb7abc1ddc878f', 'request_id': 'req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:37.229044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce40207353eb449c99cb7abc1ddc878f', 'request_id': 'req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:37.243527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] The token cache is being invalidate because project ce40207353eb449c99cb7abc1ddc878f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:37.243799 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:37.244455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:37.246054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1362/2746] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:37 2022] DELETE /identity/v3/projects/ce40207353eb449c99cb7abc1ddc878f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:38.332057 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:38.333725 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:38.350144 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:38.401426 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7c4dd-1427-424d-9aeb-a1189623e496 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:38.457116 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:38.457935 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:38.458277 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:38.458597 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:38.460657 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:38.537901 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:38.539346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:38.549422 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1385/2747] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:39.177343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:39.179169 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:39.194780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1fb1a-d84f-4ce0-84e1-b86b076744b1 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:39.200015 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c4eaf-2149-4e64-bdd6-b9c938d818dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:39.201804 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4eaf-2149-4e64-bdd6-b9c938d818dd service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:39.202206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4eaf-2149-4e64-bdd6-b9c938d818dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:39.202569 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4eaf-2149-4e64-bdd6-b9c938d818dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:39.204405 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f67c4eaf-2149-4e64-bdd6-b9c938d818dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:39.272068 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4eaf-2149-4e64-bdd6-b9c938d818dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:39.273469 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4eaf-2149-4e64-bdd6-b9c938d818dd service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:39.283972 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1363/2748] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:43.222704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:43.224816 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:43.294883 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a709e4-4fbd-46c7-a769-83b064c01439 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:43.299902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f681215-f364-4506-b442-39697da86faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:43.301093 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f681215-f364-4506-b442-39697da86faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:43.301379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f681215-f364-4506-b442-39697da86faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:43.301651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f681215-f364-4506-b442-39697da86faf admin admin] PATH_INFO: `/v3/users/b4e8a51564c04af5888e02975579c5b6` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:43.302889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f681215-f364-4506-b442-39697da86faf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e8a51564c04af5888e02975579c5b6)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:43.304827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f681215-f364-4506-b442-39697da86faf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:43.335332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:43.356738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e8a51564c04af5888e02975579c5b6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:43.368790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e8a51564c04af5888e02975579c5b6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:43.369539 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e8a51564c04af5888e02975579c5b6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:43.377902 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e8a51564c04af5888e02975579c5b6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:43.378633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e8a51564c04af5888e02975579c5b6'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:43.392287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1386/2749] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:43 2022] DELETE /identity/v3/users/b4e8a51564c04af5888e02975579c5b6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:43.397012 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:43.397513 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:43.397631 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:43.415806 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:11:43.416938 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:43.457147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:43.488245 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1364/2750] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:11:43.500206 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:43.502157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:43.567977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f681215-f364-4506-b442-39697da86faf admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:43.572287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:43.572995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:43.573288 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:43.573568 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:43.585491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:43.586852 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:43.598331 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1387/2751] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:11:43.900362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:43.901840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:43.915390 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c80d7-fd6f-485b-8af3-d310ad9b1620 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:43.920839 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:43.921838 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:43.922126 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:43.923222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] PATH_INFO: `/v3/projects/9bfdb972b07c4f90ba7dd94b122cd092` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:43.924668 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bfdb972b07c4f90ba7dd94b122cd092)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:43.927022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:43.979652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bfdb972b07c4f90ba7dd94b122cd092', 'request_id': 'req-1e83e9a0-358d-41be-a2f2-9f9012dc671e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:43.987466 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bfdb972b07c4f90ba7dd94b122cd092', 'request_id': 'req-1e83e9a0-358d-41be-a2f2-9f9012dc671e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:43.994083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bfdb972b07c4f90ba7dd94b122cd092', 'request_id': 'req-1e83e9a0-358d-41be-a2f2-9f9012dc671e'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.007104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] The token cache is being invalidate because project 9bfdb972b07c4f90ba7dd94b122cd092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:44.007531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.008046 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.009663 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1365/2752] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:43 2022] DELETE /identity/v3/projects/9bfdb972b07c4f90ba7dd94b122cd092 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:44.013832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:44.025074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563f3e-3ca0-4c6d-9b33-4d235ecef68e service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:44.096384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:44.097722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:44.098110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:44.098562 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] PATH_INFO: `/v3/users/8f13412e2b9e445b97c4985cde653a70` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:44.100418 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f13412e2b9e445b97c4985cde653a70)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:44.102798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:44.166607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f13412e2b9e445b97c4985cde653a70'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.168401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:44.188435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f13412e2b9e445b97c4985cde653a70'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.190109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f13412e2b9e445b97c4985cde653a70'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.203536 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f13412e2b9e445b97c4985cde653a70'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.204561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f13412e2b9e445b97c4985cde653a70'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.228244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1388/2753] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:44 2022] DELETE /identity/v3/users/8f13412e2b9e445b97c4985cde653a70 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:44.539502 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:44.541052 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:44.555172 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e9a0-358d-41be-a2f2-9f9012dc671e admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:44.614613 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:44.615478 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:44.615758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:44.616030 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] PATH_INFO: `/v3/projects/48bd5e9ccf8146db81631f00a1b97f0a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:44.617482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=48bd5e9ccf8146db81631f00a1b97f0a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:44.619391 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:44.666283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48bd5e9ccf8146db81631f00a1b97f0a', 'request_id': 'req-61810506-1267-4279-afcf-c4056c46c6ef'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.672175 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48bd5e9ccf8146db81631f00a1b97f0a', 'request_id': 'req-61810506-1267-4279-afcf-c4056c46c6ef'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.674295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:44.679438 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48bd5e9ccf8146db81631f00a1b97f0a', 'request_id': 'req-61810506-1267-4279-afcf-c4056c46c6ef'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.694352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] The token cache is being invalidate because project 48bd5e9ccf8146db81631f00a1b97f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:44.694641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.695275 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.696863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1366/2754] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:44 2022] DELETE /identity/v3/projects/48bd5e9ccf8146db81631f00a1b97f0a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:11:44.701595 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:44.703245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:44.716360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d68c83-045e-4563-bf7b-2b908ea6d6ac admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:44.773887 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:44.774684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:44.774973 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:44.775247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] PATH_INFO: `/v3/users/b2c5c4f811c049e9a210d7fcaf2eca58` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:44.776456 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c5c4f811c049e9a210d7fcaf2eca58)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:44.778419 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:44.825982 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c5c4f811c049e9a210d7fcaf2eca58'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.838665 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c5c4f811c049e9a210d7fcaf2eca58'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.839368 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c5c4f811c049e9a210d7fcaf2eca58'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.846970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c5c4f811c049e9a210d7fcaf2eca58'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.847575 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-23e609e4-0c04-4ce2-8931-0b1d10f9e6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c5c4f811c049e9a210d7fcaf2eca58'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:44.861565 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1389/2755] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:11:44 2022] DELETE /identity/v3/users/b2c5c4f811c049e9a210d7fcaf2eca58 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:45.156871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:11:45.158371 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:11:45.172632 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61810506-1267-4279-afcf-c4056c46c6ef admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:11:45.248310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:11:45.249225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:11:45.249510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:11:45.249791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] PATH_INFO: `/v3/projects/510dd0a5703149cc804a327525e9c4b3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:11:45.251013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510dd0a5703149cc804a327525e9c4b3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:11:45.252942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:11:45.308754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510dd0a5703149cc804a327525e9c4b3', 'request_id': 'req-f95af459-4b16-43d5-8d93-99606a060370'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:45.315455 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510dd0a5703149cc804a327525e9c4b3', 'request_id': 'req-f95af459-4b16-43d5-8d93-99606a060370'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:45.322337 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510dd0a5703149cc804a327525e9c4b3', 'request_id': 'req-f95af459-4b16-43d5-8d93-99606a060370'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:45.335760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] The token cache is being invalidate because project 510dd0a5703149cc804a327525e9c4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:11:45.336053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:45.336689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:11:45.338334 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1367/2756] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:11:45 2022] DELETE /identity/v3/projects/510dd0a5703149cc804a327525e9c4b3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:11:54.857466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=65,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:11:55.332445 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=54,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:12:21.807101 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:12:21.807684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:12:21.807684 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:12:21.826352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] MFA Rules not processed for user `4873db0f2fbf4a77a617f2102a5ccfae`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:12:21.827582 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:12:21.884046 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1390/2757] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:12:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:12:31.877295 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=25 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:06.526598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:06.527992 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:06.592901 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95af459-4b16-43d5-8d93-99606a060370 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:06.596546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:06.597239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:06.597523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:06.597802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:06.599699 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:06.674964 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:06.676682 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:06.689402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1368/2758] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:14.581705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:14.583133 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:14.595631 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035dca-0b0c-49ee-9690-3f5d9c479853 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:14.599191 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:14.599853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:14.600136 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:14.600442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:14.603563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:14.619250 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:14.620738 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:14.633589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1391/2759] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:16.683977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=54 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:24.627671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=6 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:27.301349 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:27.304047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:27.384051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08885fbe-3a5d-46bf-b45d-7cb0c5efddb5 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:27.388948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:27.390295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:27.390295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:27.390527 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] PATH_INFO: `/v3/users/4873db0f2fbf4a77a617f2102a5ccfae` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:27.391896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4873db0f2fbf4a77a617f2102a5ccfae)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:27.393827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:27.442448 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4873db0f2fbf4a77a617f2102a5ccfae'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.443104 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4873db0f2fbf4a77a617f2102a5ccfae'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.450965 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4873db0f2fbf4a77a617f2102a5ccfae'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.451643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4873db0f2fbf4a77a617f2102a5ccfae'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.464400 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-245b1eaa-87be-4c80-aa64-3386e08a10d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4873db0f2fbf4a77a617f2102a5ccfae'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.478320 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1369/2760] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:13:27 2022] DELETE /identity/v3/users/4873db0f2fbf4a77a617f2102a5ccfae => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:27.673207 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:27.685023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c8b40-97b7-4b4f-8a29-cf54451111e2 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:27.760787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:27.761629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:27.761907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:27.762180 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] PATH_INFO: `/v3/projects/a4df15f6a6dd45dab12d0b724076a23e` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:27.763389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4df15f6a6dd45dab12d0b724076a23e)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:27.765338 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:27.813992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4df15f6a6dd45dab12d0b724076a23e', 'request_id': 'req-880356ef-64da-4009-8a25-6029270b1a6e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.820604 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4df15f6a6dd45dab12d0b724076a23e', 'request_id': 'req-880356ef-64da-4009-8a25-6029270b1a6e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.828817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4df15f6a6dd45dab12d0b724076a23e', 'request_id': 'req-880356ef-64da-4009-8a25-6029270b1a6e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.840614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] The token cache is being invalidate because project a4df15f6a6dd45dab12d0b724076a23e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:13:27.840937 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.841596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-880356ef-64da-4009-8a25-6029270b1a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:27.843165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1392/2761] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:13:27 2022] DELETE /identity/v3/projects/a4df15f6a6dd45dab12d0b724076a23e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:27.848363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9655b-3b8e-4859-82af-304bdbfd0f6f None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:27.848651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9655b-3b8e-4859-82af-304bdbfd0f6f None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:27.848936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9655b-3b8e-4859-82af-304bdbfd0f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:27.869454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-5dd9655b-3b8e-4859-82af-304bdbfd0f6f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:27.870689 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5dd9655b-3b8e-4859-82af-304bdbfd0f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:27.935695 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1370/2762] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:27.943664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc574279-d5e0-4872-be7a-1387078785a3 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:27.943956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc574279-d5e0-4872-be7a-1387078785a3 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:27.944238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc574279-d5e0-4872-be7a-1387078785a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:27.969261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-bc574279-d5e0-4872-be7a-1387078785a3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:27.970524 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bc574279-d5e0-4872-be7a-1387078785a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:27.979158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1393/2763] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:27.983503 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9655b-3b8e-4859-82af-304bdbfd0f6f None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.002100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9655b-3b8e-4859-82af-304bdbfd0f6f None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.010696 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.011937 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.012392 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.012914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.014345 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:13:28.014932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.018092 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.041123 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1371/2764] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:13:28.045490 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc574279-d5e0-4872-be7a-1387078785a3 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.056744 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc574279-d5e0-4872-be7a-1387078785a3 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.061619 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.062377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.062697 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.063019 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.063839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.065845 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.095882 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1394/2765] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:13:28 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:28.100320 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.111444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5e892-2c13-4396-be9d-6fee76213dc5 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.115362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:28.182064 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.182762 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.183053 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.183330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.184033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.186044 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.208179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1372/2766] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:13:28 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:28.212803 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.223798 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eb7f9-2489-4194-9eda-2e5f7d01a338 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.229321 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.229995 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.230392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.230712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.231550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.233289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.245120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1395/2767] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:13:28.249178 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.259701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a00f2-fc23-4046-90a8-5e264ab3bd41 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.264524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ecebd-efab-4a82-a4bc-d32fc79873ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.265271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ecebd-efab-4a82-a4bc-d32fc79873ab admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.265567 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ecebd-efab-4a82-a4bc-d32fc79873ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.265977 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ecebd-efab-4a82-a4bc-d32fc79873ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.266726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ecebd-efab-4a82-a4bc-d32fc79873ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.268343 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ecebd-efab-4a82-a4bc-d32fc79873ab admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.279461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1373/2768] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:13:28.283761 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.294802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a0aa2-8d99-456a-bc03-b563f97eebbb admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.299464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.300351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.300633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.300909 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] PATH_INFO: `/v3/projects/c2499d2faee84edda5f26ec0d21aecbb/users/8f84036eb86b4907a20d46978920eadd/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.317791 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2499d2faee84edda5f26ec0d21aecbb, user_id=8f84036eb86b4907a20d46978920eadd, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.320435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.333640 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1396/2769] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:13:28 2022] PUT /identity/v3/projects/c2499d2faee84edda5f26ec0d21aecbb/users/8f84036eb86b4907a20d46978920eadd/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:28.344603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579656b-45d3-4b2a-aa16-557fcc7f762e None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.345195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579656b-45d3-4b2a-aa16-557fcc7f762e None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.345195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579656b-45d3-4b2a-aa16-557fcc7f762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.374598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-7579656b-45d3-4b2a-aa16-557fcc7f762e None None] MFA Rules not processed for user `8f84036eb86b4907a20d46978920eadd`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:28.375855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7579656b-45d3-4b2a-aa16-557fcc7f762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:28.442407 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1374/2770] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:28.454566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.456270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:28.490032 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:28.537359 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5b824-3e6c-48b2-91df-e6c9daa78d66 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.543726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41fe2b-a8d3-47b4-9ae5-dad8b4a6f489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.544379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41fe2b-a8d3-47b4-9ae5-dad8b4a6f489 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.544667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41fe2b-a8d3-47b4-9ae5-dad8b4a6f489 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.544944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41fe2b-a8d3-47b4-9ae5-dad8b4a6f489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.553459 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41fe2b-a8d3-47b4-9ae5-dad8b4a6f489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.555328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41fe2b-a8d3-47b4-9ae5-dad8b4a6f489 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.564483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1397/2771] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:28.807712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579656b-45d3-4b2a-aa16-557fcc7f762e None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:28.870707 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579656b-45d3-4b2a-aa16-557fcc7f762e None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:28.875187 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:28.876056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.876340 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.876609 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] PATH_INFO: `/v3/users/8f84036eb86b4907a20d46978920eadd` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.877821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f84036eb86b4907a20d46978920eadd)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:28.879723 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:28.896262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:28.932896 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f84036eb86b4907a20d46978920eadd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:28.933914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f84036eb86b4907a20d46978920eadd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:28.942162 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f84036eb86b4907a20d46978920eadd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:28.942869 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f84036eb86b4907a20d46978920eadd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:28.957321 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f84036eb86b4907a20d46978920eadd'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:28.971868 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1375/2772] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:13:28 2022] DELETE /identity/v3/users/8f84036eb86b4907a20d46978920eadd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:28.976354 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:28.976843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:28.976966 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:28.993932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:28.995137 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:29.057016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1398/2773] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:29.066792 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.068222 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:29.127158 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be375e-37ab-402e-a1a4-e32d8c8403eb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.130856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fefcc-b734-4c32-b84a-d2291ec03f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.131512 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fefcc-b734-4c32-b84a-d2291ec03f15 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.131796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fefcc-b734-4c32-b84a-d2291ec03f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.132662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fefcc-b734-4c32-b84a-d2291ec03f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.142489 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fefcc-b734-4c32-b84a-d2291ec03f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.143928 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fefcc-b734-4c32-b84a-d2291ec03f15 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.156060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1376/2774] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:29.437358 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.438850 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:29.454124 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c9de7-3ad1-482f-8e7e-9c3953bbd61c None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.459216 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.460131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.460596 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.460954 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] PATH_INFO: `/v3/projects/c2499d2faee84edda5f26ec0d21aecbb` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.462664 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2499d2faee84edda5f26ec0d21aecbb)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.464792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.514977 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2499d2faee84edda5f26ec0d21aecbb', 'request_id': 'req-d1228527-1ff1-4285-9664-b458e72c0fd7'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:29.520478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2499d2faee84edda5f26ec0d21aecbb', 'request_id': 'req-d1228527-1ff1-4285-9664-b458e72c0fd7'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:29.530483 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2499d2faee84edda5f26ec0d21aecbb', 'request_id': 'req-d1228527-1ff1-4285-9664-b458e72c0fd7'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:29.542925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] The token cache is being invalidate because project c2499d2faee84edda5f26ec0d21aecbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:13:29.543231 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:29.544297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-d1228527-1ff1-4285-9664-b458e72c0fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:29.545584 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1399/2775] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:13:29 2022] DELETE /identity/v3/projects/c2499d2faee84edda5f26ec0d21aecbb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:29.550428 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88cd9-8dcd-4819-aba2-bf4f7c478c83 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.550874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88cd9-8dcd-4819-aba2-bf4f7c478c83 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.551039 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88cd9-8dcd-4819-aba2-bf4f7c478c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.570506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-5fe88cd9-8dcd-4819-aba2-bf4f7c478c83 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:29.571603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5fe88cd9-8dcd-4819-aba2-bf4f7c478c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:29.631196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1377/2776] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:29.639198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717caad-f5b9-447a-9ad8-b47bea1ca769 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.639486 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717caad-f5b9-447a-9ad8-b47bea1ca769 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.639765 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717caad-f5b9-447a-9ad8-b47bea1ca769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.658184 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2717caad-f5b9-447a-9ad8-b47bea1ca769 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:29.659382 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2717caad-f5b9-447a-9ad8-b47bea1ca769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:29.667960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1400/2777] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:29.672072 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe88cd9-8dcd-4819-aba2-bf4f7c478c83 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.683060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe88cd9-8dcd-4819-aba2-bf4f7c478c83 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.688796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.689643 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.689942 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.690196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.690970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:13:29.691278 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.692959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.705212 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1378/2778] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:13:29.709622 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717caad-f5b9-447a-9ad8-b47bea1ca769 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.718911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717caad-f5b9-447a-9ad8-b47bea1ca769 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.723567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.724213 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.724501 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.724780 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.725503 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.728114 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.755122 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1401/2779] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:13:29 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:29.759421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.769485 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b6fa3-337b-4c17-8485-75b2d765f3cc admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.792577 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:29.831358 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.831952 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.832239 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.832517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.833241 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.835409 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.858133 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1379/2780] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:13:29 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:29.862443 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.873543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5bfc3-00a9-4637-a969-fd47a713f9b6 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.878350 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.879018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.879308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.879589 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.880323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.881986 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.884485 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:29.900960 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1402/2781] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:13:29.902127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.915151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7730aa-5c93-4537-9abe-fb777c48ce43 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.920070 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04191a3-d34e-4c42-85b7-92287b1b1359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.920759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04191a3-d34e-4c42-85b7-92287b1b1359 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.921090 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04191a3-d34e-4c42-85b7-92287b1b1359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.921376 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04191a3-d34e-4c42-85b7-92287b1b1359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.922119 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04191a3-d34e-4c42-85b7-92287b1b1359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.923871 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04191a3-d34e-4c42-85b7-92287b1b1359 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.937765 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1380/2782] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:13:29.944908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:29.960291 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68955837-9ff3-4b59-80ed-354b52ccc644 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:29.965158 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:29.966044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:29.966356 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:29.966629 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] PATH_INFO: `/v3/projects/aff9b900aeca4b13823a9bce6e37341c/users/73c41359673b43ad9b347ca32a6d39c4/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:29.982919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff9b900aeca4b13823a9bce6e37341c, user_id=73c41359673b43ad9b347ca32a6d39c4, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:29.985599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:29.998600 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1403/2783] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:13:29 2022] PUT /identity/v3/projects/aff9b900aeca4b13823a9bce6e37341c/users/73c41359673b43ad9b347ca32a6d39c4/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:30.008895 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f5730-8af8-4a38-b8f3-d5149ccf8aba None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.009216 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f5730-8af8-4a38-b8f3-d5149ccf8aba None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.009525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f5730-8af8-4a38-b8f3-d5149ccf8aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.036550 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-078f5730-8af8-4a38-b8f3-d5149ccf8aba None None] MFA Rules not processed for user `73c41359673b43ad9b347ca32a6d39c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:30.037789 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-078f5730-8af8-4a38-b8f3-d5149ccf8aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:30.109085 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1381/2784] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:30.113493 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.180287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7750fe-ce70-4587-a24a-6d6a193cecc2 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.184992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.185662 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.185953 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.186234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.186925 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.188824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.218873 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1404/2785] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:13:30 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:30.223870 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f5730-8af8-4a38-b8f3-d5149ccf8aba None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.237132 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f5730-8af8-4a38-b8f3-d5149ccf8aba None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.296223 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.296906 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.297240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.297506 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.298207 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.300160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.322401 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1382/2786] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:13:30 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:30.326491 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.339010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4ae2a-6b62-418e-9c2d-1cf8e27891cf admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.343965 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.344661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.344944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.345247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.345980 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.347579 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.359824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1405/2787] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:13:30.363966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.374281 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e2d78-965b-4779-bf89-4f8414c54ec9 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.379060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.379948 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.380231 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.380507 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] PATH_INFO: `/v3/projects/af30716dcb9341bd8d951ee1205cd727/users/6c7e5bdfe0b6484cb2675a08d33086f3/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.396237 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=af30716dcb9341bd8d951ee1205cd727, user_id=6c7e5bdfe0b6484cb2675a08d33086f3, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.400028 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.409746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:30.415342 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1383/2788] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:13:30 2022] PUT /identity/v3/projects/af30716dcb9341bd8d951ee1205cd727/users/6c7e5bdfe0b6484cb2675a08d33086f3/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:30.419340 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.432273 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cd4d0-35dd-4f80-87b1-558a18bf3fe1 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.494278 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a19f3-31c5-45ec-99ec-e2d19889a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.494964 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a19f3-31c5-45ec-99ec-e2d19889a2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.495249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a19f3-31c5-45ec-99ec-e2d19889a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.495528 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a19f3-31c5-45ec-99ec-e2d19889a2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.496268 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a19f3-31c5-45ec-99ec-e2d19889a2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.497918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a19f3-31c5-45ec-99ec-e2d19889a2b2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.509167 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1406/2789] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:13:30.513625 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.524995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e057111-c358-4983-b21a-4d21d8c572eb admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.529907 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.530876 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.531156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.531434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] PATH_INFO: `/v3/projects/af30716dcb9341bd8d951ee1205cd727/users/6c7e5bdfe0b6484cb2675a08d33086f3/roles/fe0d00714612495091430762d114121a` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.533452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af30716dcb9341bd8d951ee1205cd727, user_id=6c7e5bdfe0b6484cb2675a08d33086f3, role_id=fe0d00714612495091430762d114121a)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.536148 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.547581 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1384/2790] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:13:30 2022] PUT /identity/v3/projects/af30716dcb9341bd8d951ee1205cd727/users/6c7e5bdfe0b6484cb2675a08d33086f3/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:30.567391 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a78089-0380-4c59-9b1b-9ca6fde2cbc1 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.567745 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a78089-0380-4c59-9b1b-9ca6fde2cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.568141 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a78089-0380-4c59-9b1b-9ca6fde2cbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.598482 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-73a78089-0380-4c59-9b1b-9ca6fde2cbc1 None None] MFA Rules not processed for user `6c7e5bdfe0b6484cb2675a08d33086f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:30.599701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-73a78089-0380-4c59-9b1b-9ca6fde2cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:30.630854 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:30.671922 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1407/2791] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:30.683714 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.685629 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:30.749711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f0918-552a-4f03-93e4-c7563bcfec40 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.753557 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.754240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.754523 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.754796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.765236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.766648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.776235 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1385/2792] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:30.850289 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a78089-0380-4c59-9b1b-9ca6fde2cbc1 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:30.860870 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a78089-0380-4c59-9b1b-9ca6fde2cbc1 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:30.864520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbfa5d-88b2-4e33-be5b-90a8ab10eb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:30.865188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbfa5d-88b2-4e33-be5b-90a8ab10eb1f service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:30.865475 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbfa5d-88b2-4e33-be5b-90a8ab10eb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:30.865756 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbfa5d-88b2-4e33-be5b-90a8ab10eb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:30.923300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbfa5d-88b2-4e33-be5b-90a8ab10eb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:30.924653 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbfa5d-88b2-4e33-be5b-90a8ab10eb1f service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:30.934299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1408/2793] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:40.221364 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:40.288019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cde2a0-869c-4796-badb-8f81d8266165 service cinder] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:40.293592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:40.294499 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:40.294791 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:40.295060 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] PATH_INFO: `/v3/users/73c41359673b43ad9b347ca32a6d39c4` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:40.296249 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c41359673b43ad9b347ca32a6d39c4)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:40.298210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:40.349427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c41359673b43ad9b347ca32a6d39c4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.350147 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c41359673b43ad9b347ca32a6d39c4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.358352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c41359673b43ad9b347ca32a6d39c4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.359087 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c41359673b43ad9b347ca32a6d39c4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.373481 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c41359673b43ad9b347ca32a6d39c4'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.388174 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1386/2794] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:13:40 2022] DELETE /identity/v3/users/73c41359673b43ad9b347ca32a6d39c4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:40.392796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:40.393421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:40.393421 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:40.412638 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:40.413830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:40.481479 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1409/2795] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:13:40.491802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:40.493844 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:40.503879 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:40.557166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72a5e9-0860-4d67-b453-b285af97ac7d admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:40.560829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:40.561510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:40.561793 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:40.562076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:40.572062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:40.573536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:40.583571 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1387/2796] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:40.884612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:40.886077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:40.898430 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea4aa-0e62-470e-8319-1eeec886273f None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:40.903681 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:40.904560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:40.904843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:40.905155 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] PATH_INFO: `/v3/projects/aff9b900aeca4b13823a9bce6e37341c` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:40.906370 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff9b900aeca4b13823a9bce6e37341c)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:40.908324 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:40.967218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff9b900aeca4b13823a9bce6e37341c', 'request_id': 'req-bd581da8-29a0-4d95-87cf-d20cada7300c'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.972541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff9b900aeca4b13823a9bce6e37341c', 'request_id': 'req-bd581da8-29a0-4d95-87cf-d20cada7300c'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.975300 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:40.981476 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff9b900aeca4b13823a9bce6e37341c', 'request_id': 'req-bd581da8-29a0-4d95-87cf-d20cada7300c'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.994969 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] The token cache is being invalidate because project aff9b900aeca4b13823a9bce6e37341c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:13:40.995314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.995932 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:40.997557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1410/2797] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:13:40 2022] DELETE /identity/v3/projects/aff9b900aeca4b13823a9bce6e37341c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:41.001805 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:41.011398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ec7ce-d9fb-4f7f-9beb-ffd993be78bf service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:41.067603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:41.068477 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:41.069010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:41.069303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] PATH_INFO: `/v3/users/6c7e5bdfe0b6484cb2675a08d33086f3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:41.070504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c7e5bdfe0b6484cb2675a08d33086f3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:41.072417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:41.118944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c7e5bdfe0b6484cb2675a08d33086f3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.119607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c7e5bdfe0b6484cb2675a08d33086f3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.126406 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c7e5bdfe0b6484cb2675a08d33086f3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.127019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c7e5bdfe0b6484cb2675a08d33086f3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.139385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-b241809d-9322-419b-b994-e98741b529a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c7e5bdfe0b6484cb2675a08d33086f3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.155228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1388/2798] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:13:41 2022] DELETE /identity/v3/users/6c7e5bdfe0b6484cb2675a08d33086f3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:13:41.472772 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:41.474341 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:41.494316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd581da8-29a0-4d95-87cf-d20cada7300c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:41.575523 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:41.576647 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:41.576918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:41.577235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] PATH_INFO: `/v3/projects/af30716dcb9341bd8d951ee1205cd727` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:41.578519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af30716dcb9341bd8d951ee1205cd727)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:41.580478 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:41.644599 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af30716dcb9341bd8d951ee1205cd727', 'request_id': 'req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.651529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af30716dcb9341bd8d951ee1205cd727', 'request_id': 'req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.661894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af30716dcb9341bd8d951ee1205cd727', 'request_id': 'req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.676505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] The token cache is being invalidate because project af30716dcb9341bd8d951ee1205cd727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:13:41.677083 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.677770 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-315ce8fc-0ca7-4a30-b4e8-251ba19fbfc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:41.679348 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1411/2799] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:13:41 2022] DELETE /identity/v3/projects/af30716dcb9341bd8d951ee1205cd727 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:13:41.684433 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56a79-6a70-4819-aa7b-0d273faefb53 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:41.685069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56a79-6a70-4819-aa7b-0d273faefb53 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:41.685069 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56a79-6a70-4819-aa7b-0d273faefb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:41.707742 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8fd56a79-6a70-4819-aa7b-0d273faefb53 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:41.709536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8fd56a79-6a70-4819-aa7b-0d273faefb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:41.739365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:41.783474 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1389/2800] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:41.793790 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e3849-7699-4940-a636-be1b21a64e56 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:41.794263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e3849-7699-4940-a636-be1b21a64e56 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:41.794739 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e3849-7699-4940-a636-be1b21a64e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:41.816131 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-c02e3849-7699-4940-a636-be1b21a64e56 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:41.817400 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c02e3849-7699-4940-a636-be1b21a64e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:41.827024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1412/2801] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:41.831615 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56a79-6a70-4819-aa7b-0d273faefb53 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:41.846821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56a79-6a70-4819-aa7b-0d273faefb53 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:41.852960 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:41.853688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:41.854125 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:41.854623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:41.855846 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:13:41.856164 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:41.858397 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:41.873720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1390/2802] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:13:41.879198 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e3849-7699-4940-a636-be1b21a64e56 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:41.893343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e3849-7699-4940-a636-be1b21a64e56 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:41.898401 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:41.899084 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:41.899367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:41.899643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:41.900543 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:41.902571 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:41.932208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1413/2803] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:13:41 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:41.937305 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:41.957528 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b743da-b397-4e79-8d87-7ae283c108fe admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:42.041768 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:42.042454 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:42.042738 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:42.043013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:42.043815 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:42.045761 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:42.070463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1391/2804] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:13:41 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:13:42.075730 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:42.089076 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c9430-0fcc-4909-af47-280350e00b02 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:42.093831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:42.094530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:42.094817 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:42.095103 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:42.095847 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:42.098671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:42.111271 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1414/2805] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:13:42.115460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:42.128560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89095-cd32-45ae-846f-70776e5e1c29 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:42.133327 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f8081-ed42-4127-8709-5adcad6425d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:42.134047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8081-ed42-4127-8709-5adcad6425d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:42.134333 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8081-ed42-4127-8709-5adcad6425d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:42.134607 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8081-ed42-4127-8709-5adcad6425d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:42.135378 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f8081-ed42-4127-8709-5adcad6425d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:42.137000 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f8081-ed42-4127-8709-5adcad6425d9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:42.148025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1392/2806] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:13:42.153218 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:42.163808 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6e276-a0b9-43a1-8012-3c7797eb4bd7 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:42.169442 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:42.170333 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:42.170616 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:42.170891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] PATH_INFO: `/v3/projects/516772f9d6014a6b9a9883db97d1b237/users/0aa886455fd5470a9fcb855c7a8f9fc3/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:42.187567 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=516772f9d6014a6b9a9883db97d1b237, user_id=0aa886455fd5470a9fcb855c7a8f9fc3, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:42.190473 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:42.209044 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1415/2807] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:13:42 2022] PUT /identity/v3/projects/516772f9d6014a6b9a9883db97d1b237/users/0aa886455fd5470a9fcb855c7a8f9fc3/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:42.220746 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22a3d-69e5-4652-968c-7afa95d8fd11 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:42.221213 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22a3d-69e5-4652-968c-7afa95d8fd11 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:42.221592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22a3d-69e5-4652-968c-7afa95d8fd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:42.251531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-9cb22a3d-69e5-4652-968c-7afa95d8fd11 None None] MFA Rules not processed for user `0aa886455fd5470a9fcb855c7a8f9fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:42.252788 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-9cb22a3d-69e5-4652-968c-7afa95d8fd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:42.327524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1393/2808] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:42.339031 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:42.340899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:42.360398 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:42.412636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7d4eb-d255-4ff2-ac2c-8671b2694a6b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:42.416703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69493026-175b-42eb-a5e0-7e37a266ef35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d3144975804e369a023316655be2d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:42.417494 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493026-175b-42eb-a5e0-7e37a266ef35 service glance] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:42.417911 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493026-175b-42eb-a5e0-7e37a266ef35 service glance] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:42.418279 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493026-175b-42eb-a5e0-7e37a266ef35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:42.429636 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69493026-175b-42eb-a5e0-7e37a266ef35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:42.431142 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69493026-175b-42eb-a5e0-7e37a266ef35 service glance] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:42.441072 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1416/2809] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:13:52.324412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:52.436392 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=19 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:59.071802 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22a3d-69e5-4652-968c-7afa95d8fd11 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:59.140228 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22a3d-69e5-4652-968c-7afa95d8fd11 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:59.144701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:59.145603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:59.145886 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:59.146163 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] PATH_INFO: `/v3/users/0aa886455fd5470a9fcb855c7a8f9fc3` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:59.147356 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa886455fd5470a9fcb855c7a8f9fc3)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:59.149303 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:59.195915 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa886455fd5470a9fcb855c7a8f9fc3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.196603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa886455fd5470a9fcb855c7a8f9fc3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.204127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa886455fd5470a9fcb855c7a8f9fc3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.204767 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa886455fd5470a9fcb855c7a8f9fc3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.217672 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa886455fd5470a9fcb855c7a8f9fc3'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.231115 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1394/2810] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:13:59 2022] DELETE /identity/v3/users/0aa886455fd5470a9fcb855c7a8f9fc3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:59.235719 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:59.236234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:59.236343 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:59.253538 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:59.254755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:59.316226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1417/2811] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:59.326662 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:59.328088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:59.384127 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863e092-bf7e-49ae-beea-92e1da4fb4b1 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:59.388602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8aee2-7723-4e90-a458-ca6472e723ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:59.389271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8aee2-7723-4e90-a458-ca6472e723ad service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:59.389557 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8aee2-7723-4e90-a458-ca6472e723ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:59.389856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8aee2-7723-4e90-a458-ca6472e723ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:59.398782 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8aee2-7723-4e90-a458-ca6472e723ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:59.400220 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8aee2-7723-4e90-a458-ca6472e723ad service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:59.410344 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1395/2812] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:13:59.686345 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:13:59.687855 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:59.705361 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35873d4e-016e-4207-b2de-d8fa52ca0e1f None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:13:59.710118 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:13:59.711010 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:59.711326 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:59.711843 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] PATH_INFO: `/v3/projects/516772f9d6014a6b9a9883db97d1b237` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:59.713286 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] RBAC: Authorizing `identity:delete_project(project_id=516772f9d6014a6b9a9883db97d1b237)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:13:59.716304 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:13:59.767864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516772f9d6014a6b9a9883db97d1b237', 'request_id': 'req-4956786e-e2ce-49cc-b8d5-7fe83203748e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.773410 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516772f9d6014a6b9a9883db97d1b237', 'request_id': 'req-4956786e-e2ce-49cc-b8d5-7fe83203748e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.782676 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516772f9d6014a6b9a9883db97d1b237', 'request_id': 'req-4956786e-e2ce-49cc-b8d5-7fe83203748e'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.801177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] The token cache is being invalidate because project 516772f9d6014a6b9a9883db97d1b237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:13:59.801455 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.802374 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-4956786e-e2ce-49cc-b8d5-7fe83203748e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:13:59.803942 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1418/2813] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:13:59 2022] DELETE /identity/v3/projects/516772f9d6014a6b9a9883db97d1b237 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:13:59.809434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302d25-1977-4d96-82cb-1d19ff8635b8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:59.809739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302d25-1977-4d96-82cb-1d19ff8635b8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:59.810008 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302d25-1977-4d96-82cb-1d19ff8635b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:59.830892 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-ca302d25-1977-4d96-82cb-1d19ff8635b8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:59.832102 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-ca302d25-1977-4d96-82cb-1d19ff8635b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:59.850091 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:13:59.896648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1396/2814] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:59.959051 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c249d-aee7-4b5a-a049-679d96ec8516 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:13:59.959367 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c249d-aee7-4b5a-a049-679d96ec8516 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:13:59.959677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c249d-aee7-4b5a-a049-679d96ec8516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:13:59.979445 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-670c249d-aee7-4b5a-a049-679d96ec8516 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:13:59.981802 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-670c249d-aee7-4b5a-a049-679d96ec8516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:13:59.993703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1419/2815] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:13:59.999313 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302d25-1977-4d96-82cb-1d19ff8635b8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.010199 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302d25-1977-4d96-82cb-1d19ff8635b8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.016466 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.017641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.018180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.018702 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.019959 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:14:00.020444 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.023209 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.038574 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1397/2816] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:14:00.043771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c249d-aee7-4b5a-a049-679d96ec8516 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.053752 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c249d-aee7-4b5a-a049-679d96ec8516 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.060276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.060944 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.061260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.061541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.062397 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.064212 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.092308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1420/2817] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:14:00 2022] POST /identity/v3/projects => generated 359 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:14:00.097452 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.111975 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf610d-2911-43c2-8b28-81d9e0736d80 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.171365 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.171978 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.172262 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.172536 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.173269 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.175173 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.198077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1398/2818] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:14:00 2022] POST /identity/v3/users => generated 280 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:14:00.202112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.213033 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b882-0aa1-44b0-86e8-a47636aad572 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.219352 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.220466 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.220781 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.221217 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.222096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.224437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.237048 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1421/2819] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:14:00.241272 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.252051 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e8545-dd63-45dc-a23e-7b5160f75541 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.256592 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d6692-a35d-4bcd-9dc5-bd8db5a698fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.257282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d6692-a35d-4bcd-9dc5-bd8db5a698fa admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.257597 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d6692-a35d-4bcd-9dc5-bd8db5a698fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.257841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d6692-a35d-4bcd-9dc5-bd8db5a698fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.258647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d6692-a35d-4bcd-9dc5-bd8db5a698fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.260168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d6692-a35d-4bcd-9dc5-bd8db5a698fa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.270456 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1399/2820] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:14:00.274987 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.285703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7ec6b-ab6f-4a94-b792-2b1ee460b60b admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.291788 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.293366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.293645 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.293916 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] PATH_INFO: `/v3/projects/bbc5620180c74e84987c119eff8a6d8f/users/b9560a928af4418bbe82b39e684f79a5/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.310247 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc5620180c74e84987c119eff8a6d8f, user_id=b9560a928af4418bbe82b39e684f79a5, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.312992 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.326900 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1422/2821] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:14:00 2022] PUT /identity/v3/projects/bbc5620180c74e84987c119eff8a6d8f/users/b9560a928af4418bbe82b39e684f79a5/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:00.336971 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4569a-8cae-41f3-9315-aca3d60ba2ae None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.337290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4569a-8cae-41f3-9315-aca3d60ba2ae None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.337573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4569a-8cae-41f3-9315-aca3d60ba2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.364861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-17f4569a-8cae-41f3-9315-aca3d60ba2ae None None] MFA Rules not processed for user `b9560a928af4418bbe82b39e684f79a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:00.366100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-17f4569a-8cae-41f3-9315-aca3d60ba2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:00.430808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1400/2822] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:00 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:00.460778 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.462497 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:00.485066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:00.523297 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabad216-c42f-423e-a86d-9f2bd40d65dc admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.527750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc383e59-0290-4e7b-8508-8feda8311f29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.528115 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc383e59-0290-4e7b-8508-8feda8311f29 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.528399 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc383e59-0290-4e7b-8508-8feda8311f29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.528677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc383e59-0290-4e7b-8508-8feda8311f29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.536994 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc383e59-0290-4e7b-8508-8feda8311f29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.538438 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc383e59-0290-4e7b-8508-8feda8311f29 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.547246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1423/2823] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:00 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:00.561973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4569a-8cae-41f3-9315-aca3d60ba2ae None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.581914 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:00.627847 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4569a-8cae-41f3-9315-aca3d60ba2ae None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.632451 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.633352 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.633640 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.633918 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] PATH_INFO: `/v3/users/b9560a928af4418bbe82b39e684f79a5` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.635124 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9560a928af4418bbe82b39e684f79a5)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.637944 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.686350 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9560a928af4418bbe82b39e684f79a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:00.686931 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9560a928af4418bbe82b39e684f79a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:00.693947 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9560a928af4418bbe82b39e684f79a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:00.694602 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9560a928af4418bbe82b39e684f79a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:00.706626 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9560a928af4418bbe82b39e684f79a5'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:00.719533 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1401/2824] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:14:00 2022] DELETE /identity/v3/users/b9560a928af4418bbe82b39e684f79a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:00.724222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.724515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.724801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.742637 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:00.743894 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:00.823281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1424/2825] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:00.835047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:00.837875 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:00.920256 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18e38b-bced-4611-8687-639b2dd62a29 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:00.924832 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceae993-2308-42e6-87e6-4e37ce4541ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:00.925966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceae993-2308-42e6-87e6-4e37ce4541ad service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:00.926461 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceae993-2308-42e6-87e6-4e37ce4541ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:00.926827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceae993-2308-42e6-87e6-4e37ce4541ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:00.940465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceae993-2308-42e6-87e6-4e37ce4541ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:00.942331 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceae993-2308-42e6-87e6-4e37ce4541ad service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:00.953416 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1402/2826] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:01.238515 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.240325 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:01.254067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac45fdf-b925-4c27-aea8-0ac901b767be None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.258962 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.259844 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.260129 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.260408 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] PATH_INFO: `/v3/projects/bbc5620180c74e84987c119eff8a6d8f` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.262413 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc5620180c74e84987c119eff8a6d8f)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.264038 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.313468 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc5620180c74e84987c119eff8a6d8f', 'request_id': 'req-5472be3f-c739-4333-806c-ee035a06e248'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:01.319950 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc5620180c74e84987c119eff8a6d8f', 'request_id': 'req-5472be3f-c739-4333-806c-ee035a06e248'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:01.327502 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc5620180c74e84987c119eff8a6d8f', 'request_id': 'req-5472be3f-c739-4333-806c-ee035a06e248'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:01.339773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] The token cache is being invalidate because project bbc5620180c74e84987c119eff8a6d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:14:01.340557 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:01.340699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-5472be3f-c739-4333-806c-ee035a06e248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:01.342246 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1425/2827] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:14:01 2022] DELETE /identity/v3/projects/bbc5620180c74e84987c119eff8a6d8f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:14:01.347384 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93aaef-d8e9-4e29-bb31-54eea43c42b8 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.347855 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93aaef-d8e9-4e29-bb31-54eea43c42b8 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.347969 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93aaef-d8e9-4e29-bb31-54eea43c42b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.367099 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2e93aaef-d8e9-4e29-bb31-54eea43c42b8 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:01.368308 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2e93aaef-d8e9-4e29-bb31-54eea43c42b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:01.403784 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:01.432258 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1403/2828] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:01.441332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221749f3-6d6d-463a-aa73-e1f240dbd451 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.441624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221749f3-6d6d-463a-aa73-e1f240dbd451 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.441912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221749f3-6d6d-463a-aa73-e1f240dbd451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.460499 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-221749f3-6d6d-463a-aa73-e1f240dbd451 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:01.461740 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-221749f3-6d6d-463a-aa73-e1f240dbd451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:01.470244 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1426/2829] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:01.474408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93aaef-d8e9-4e29-bb31-54eea43c42b8 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.484827 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93aaef-d8e9-4e29-bb31-54eea43c42b8 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.489995 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.490679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.490970 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.491254 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.492022 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:14:01.492335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.493973 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.506310 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1404/2830] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:14:01.510677 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221749f3-6d6d-463a-aa73-e1f240dbd451 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.521069 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221749f3-6d6d-463a-aa73-e1f240dbd451 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.525809 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.526460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.526804 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.527708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.528414 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.530347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.565030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1427/2831] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:14:01 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:14:01.569402 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.580062 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06184155-1a43-4ce6-95a6-709c8752b50a admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.637510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.638195 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.638492 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.638779 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.639482 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.641434 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.661511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1405/2832] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:14:01 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:14:01.665727 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.676119 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facdb7a-7f29-4bc6-8424-b135b00380b1 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.680910 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.681609 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.681888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.682159 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.682908 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.684728 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.695007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1428/2833] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:14:01.699541 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.709457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4600cd8-fe19-441f-b883-ccb1ca34f2de admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.714100 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5c3f9-105f-4ed8-861a-564870fd7649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.715467 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c3f9-105f-4ed8-861a-564870fd7649 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.715753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c3f9-105f-4ed8-861a-564870fd7649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.716031 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c3f9-105f-4ed8-861a-564870fd7649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.716759 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5c3f9-105f-4ed8-861a-564870fd7649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.718385 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5c3f9-105f-4ed8-861a-564870fd7649 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.728338 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1406/2834] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:14:01.732722 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.742793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99739ca2-f2a1-43bc-aa20-d4a36f42a90e admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.748022 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.748897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.749215 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.749495 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] PATH_INFO: `/v3/projects/970b5fbd6c22482db1dbab88ab6159b8/users/74dad5301f184a8191d2974ab5cbfc28/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.751801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:01.766840 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=970b5fbd6c22482db1dbab88ab6159b8, user_id=74dad5301f184a8191d2974ab5cbfc28, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.769527 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.785078 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1429/2835] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:14:01 2022] PUT /identity/v3/projects/970b5fbd6c22482db1dbab88ab6159b8/users/74dad5301f184a8191d2974ab5cbfc28/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:01.795414 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4822-485a-479a-bd45-d0e42ee7748d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.795705 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4822-485a-479a-bd45-d0e42ee7748d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.795988 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4822-485a-479a-bd45-d0e42ee7748d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.823572 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-0d7f4822-485a-479a-bd45-d0e42ee7748d None None] MFA Rules not processed for user `74dad5301f184a8191d2974ab5cbfc28`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:01.825493 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-0d7f4822-485a-479a-bd45-d0e42ee7748d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:01.891065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1407/2836] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:01.902477 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:01.904381 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:01.964376 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51259-d92c-49b5-a8d3-d73739efb29a admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:01.968238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78628f3-61f4-4c96-9de7-118ba92954ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:01.968877 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78628f3-61f4-4c96-9de7-118ba92954ca service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:01.969193 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78628f3-61f4-4c96-9de7-118ba92954ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:01.969469 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78628f3-61f4-4c96-9de7-118ba92954ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:01.978030 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78628f3-61f4-4c96-9de7-118ba92954ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:01.979460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78628f3-61f4-4c96-9de7-118ba92954ca service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:01.989208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1430/2837] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:11.888136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:11.984593 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:36.005590 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f4822-485a-479a-bd45-d0e42ee7748d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:36.068603 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f4822-485a-479a-bd45-d0e42ee7748d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:36.073443 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:36.074322 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.074610 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.074888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] PATH_INFO: `/v3/users/74dad5301f184a8191d2974ab5cbfc28` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.076097 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74dad5301f184a8191d2974ab5cbfc28)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:36.078065 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:36.125647 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74dad5301f184a8191d2974ab5cbfc28'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.126295 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74dad5301f184a8191d2974ab5cbfc28'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.133025 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74dad5301f184a8191d2974ab5cbfc28'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.133658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74dad5301f184a8191d2974ab5cbfc28'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.145752 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74dad5301f184a8191d2974ab5cbfc28'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.159504 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1408/2838] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:14:36 2022] DELETE /identity/v3/users/74dad5301f184a8191d2974ab5cbfc28 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:36.164056 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dddce-f399-498c-ad41-3927f5720166 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.164560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dddce-f399-498c-ad41-3927f5720166 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.164667 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dddce-f399-498c-ad41-3927f5720166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.182240 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-556dddce-f399-498c-ad41-3927f5720166 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:36.183435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-556dddce-f399-498c-ad41-3927f5720166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:36.242405 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1431/2839] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:36.252585 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:36.254679 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:36.313179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea13255-f879-4e30-b595-3b2739347577 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:36.317630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb377a-6677-4d64-a6b8-1fc00192824e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:36.318282 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb377a-6677-4d64-a6b8-1fc00192824e service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.318570 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb377a-6677-4d64-a6b8-1fc00192824e service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.318856 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb377a-6677-4d64-a6b8-1fc00192824e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.328152 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb377a-6677-4d64-a6b8-1fc00192824e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:36.329619 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb377a-6677-4d64-a6b8-1fc00192824e service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:36.339047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1409/2840] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:36.612824 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dddce-f399-498c-ad41-3927f5720166 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:36.614556 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-556dddce-f399-498c-ad41-3927f5720166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:36.627773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dddce-f399-498c-ad41-3927f5720166 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:36.632444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:36.633351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.633641 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.634065 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] PATH_INFO: `/v3/projects/970b5fbd6c22482db1dbab88ab6159b8` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.635274 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=970b5fbd6c22482db1dbab88ab6159b8)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:36.637222 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:36.686203 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '970b5fbd6c22482db1dbab88ab6159b8', 'request_id': 'req-ac8cb9af-0e45-467c-9953-2d233280fa48'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.692000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '970b5fbd6c22482db1dbab88ab6159b8', 'request_id': 'req-ac8cb9af-0e45-467c-9953-2d233280fa48'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.699573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '970b5fbd6c22482db1dbab88ab6159b8', 'request_id': 'req-ac8cb9af-0e45-467c-9953-2d233280fa48'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.710746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] The token cache is being invalidate because project 970b5fbd6c22482db1dbab88ab6159b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80107) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:14:36.711026 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.711652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-ac8cb9af-0e45-467c-9953-2d233280fa48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:36.713234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1432/2841] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:14:36 2022] DELETE /identity/v3/projects/970b5fbd6c22482db1dbab88ab6159b8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:36.718080 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a37c25-1d86-4807-8564-e763c2c77bf5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.718374 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a37c25-1d86-4807-8564-e763c2c77bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.718658 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a37c25-1d86-4807-8564-e763c2c77bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.737623 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-b1a37c25-1d86-4807-8564-e763c2c77bf5 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:36.738845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-b1a37c25-1d86-4807-8564-e763c2c77bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:36.755465 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:36.798168 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1410/2842] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:36.806749 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e789d-9f88-475a-a69d-2c53a18b0f6b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.807045 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e789d-9f88-475a-a69d-2c53a18b0f6b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.807328 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e789d-9f88-475a-a69d-2c53a18b0f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.825965 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-0e7e789d-9f88-475a-a69d-2c53a18b0f6b None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:36.827223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0e7e789d-9f88-475a-a69d-2c53a18b0f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:36.835508 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1433/2843] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:36.839755 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a37c25-1d86-4807-8564-e763c2c77bf5 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:36.850179 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a37c25-1d86-4807-8564-e763c2c77bf5 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:36.854760 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:36.855420 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.855710 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.855993 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.856729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80108) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:14:36.857076 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:36.858687 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:36.871377 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1411/2844] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:14:36.875763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e789d-9f88-475a-a69d-2c53a18b0f6b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:36.885226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e789d-9f88-475a-a69d-2c53a18b0f6b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:36.889827 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:36.890471 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.890748 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.891017 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] PATH_INFO: `/v3/projects` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.891705 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:36.893714 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:36.919447 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1434/2845] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:14:36 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:14:36.923510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:36.934346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98964817-79f6-4c23-9f34-4ea4e1f25b52 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:36.994532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:36.995210 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:36.995487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:36.995758 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:36.996450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:36.998373 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:37.019435 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1412/2846] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:14:36 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:14:37.024612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:37.035630 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3627a-d3a3-4d4a-8762-3b3dd558d779 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:37.040174 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:37.040874 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:37.041189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:37.041465 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:37.042189 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:37.043785 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:37.055617 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1435/2847] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:14:37.059641 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:37.069859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ecfe-9a2a-4a7f-a93a-79063cea03b6 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:37.074462 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de50bdc-398b-4b97-ac6e-77021ac21474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:37.075141 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de50bdc-398b-4b97-ac6e-77021ac21474 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:37.075408 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de50bdc-398b-4b97-ac6e-77021ac21474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:37.075688 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de50bdc-398b-4b97-ac6e-77021ac21474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:37.076418 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de50bdc-398b-4b97-ac6e-77021ac21474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:37.078041 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de50bdc-398b-4b97-ac6e-77021ac21474 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:37.088463 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1413/2848] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:14:37.092474 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:37.103110 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cc75c-850a-4af3-ae90-c9f0bd6bc02d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:37.107643 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:37.108507 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:37.108787 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:37.109096 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] PATH_INFO: `/v3/projects/74c7b1ba386348c0b22d4cc4bf296735/users/0bf77869a8f84d9ca61053e6a0fc08d9/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:37.124186 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c7b1ba386348c0b22d4cc4bf296735, user_id=0bf77869a8f84d9ca61053e6a0fc08d9, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:37.127551 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:37.140460 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1436/2849] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:14:37 2022] PUT /identity/v3/projects/74c7b1ba386348c0b22d4cc4bf296735/users/0bf77869a8f84d9ca61053e6a0fc08d9/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:37.151621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d357ef-ed06-428b-8546-287c990df1f4 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:37.151997 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d357ef-ed06-428b-8546-287c990df1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:37.152203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d357ef-ed06-428b-8546-287c990df1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:37.304160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-44d357ef-ed06-428b-8546-287c990df1f4 None None] MFA Rules not processed for user `0bf77869a8f84d9ca61053e6a0fc08d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:37.305460 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-44d357ef-ed06-428b-8546-287c990df1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:37.370078 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1414/2850] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:37.381351 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:37.383039 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:37.405235 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:37.445332 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4fdeb-1b12-4b12-bb64-c8d2fd856412 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:37.449005 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:37.449666 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:37.449951 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:37.450229 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:37.459834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:37.461310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:37.470605 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1437/2851] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:38.818883 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:38.819429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:38.819429 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:38.837648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] MFA Rules not processed for user `0bf77869a8f84d9ca61053e6a0fc08d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:38.838841 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:38.845650 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1415/2852] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:39.344458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:39.346135 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:39.417860 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf9688-1dc8-468e-b6a8-828920c013b4 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:39.422915 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:39.423559 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:39.424433 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:39.424671 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:39.426625 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:39.438314 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:39.440668 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:39.450594 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1438/2853] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:41.601267 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:41.602865 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:41.619043 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:41.668399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653019a-6759-46e2-88a6-f1e0a6710119 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:41.672129 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:41.672888 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:41.673270 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:41.673611 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:41.678778 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:41.689880 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:41.691909 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:41.700562 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1416/2854] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:42.363148 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:42.364853 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:42.465834 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6fe16-d74b-43ee-81f5-5a0b9850b733 service placement] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:42.475282 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c7f72-ba1e-4cbe-8156-8414302be981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:42.475989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7f72-ba1e-4cbe-8156-8414302be981 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:42.476270 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7f72-ba1e-4cbe-8156-8414302be981 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:42.476547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7f72-ba1e-4cbe-8156-8414302be981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:42.478514 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-1f8c7f72-ba1e-4cbe-8156-8414302be981 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:42.510547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c7f72-ba1e-4cbe-8156-8414302be981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:42.510547 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c7f72-ba1e-4cbe-8156-8414302be981 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:42.524379 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1439/2855] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:47.903648 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:47.918898 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe7021-4334-470c-b791-586df77b4876 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:47.924595 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:47.926038 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:47.926412 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:47.926694 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:47.928808 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:47.943050 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:47.944346 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:47.955646 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1417/2856] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:52.515387 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=83 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:53.725429 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d4887-f437-45c6-9937-0adab2226aa2 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:53.726090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d4887-f437-45c6-9937-0adab2226aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:53.726090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d4887-f437-45c6-9937-0adab2226aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:53.743280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-8f4d4887-f437-45c6-9937-0adab2226aa2 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:53.744384 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-8f4d4887-f437-45c6-9937-0adab2226aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:53.798651 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1440/2857] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:14:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:53.846563 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:53.848120 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:53.915608 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938e096-78d9-4c4e-9d5b-f5fae131b648 service nova] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:53.924431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:53.924431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:53.924431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:53.924431 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:53.933047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:53.934621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:53.943161 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1418/2858] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:14:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:14:54.478112 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df3359-9381-4f85-9232-e37092e55ea5 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:54.478661 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df3359-9381-4f85-9232-e37092e55ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:54.479670 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df3359-9381-4f85-9232-e37092e55ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:54.500489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-b7df3359-9381-4f85-9232-e37092e55ea5 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:54.501868 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-b7df3359-9381-4f85-9232-e37092e55ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:54.509378 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1441/2859] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jun 29 08:14:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:54.522058 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:54.533026 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54828b7f-0a0a-4332-af98-cfebe1eab844 service swift] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:54.538877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:54.539532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:54.539798 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:54.540077 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:54.548652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:54.550701 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:54.560010 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1419/2860] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:14:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:14:54.748280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15987c49-9cc5-44ff-b384-a7b7ec3a49c7 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:54.748771 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15987c49-9cc5-44ff-b384-a7b7ec3a49c7 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:54.748891 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15987c49-9cc5-44ff-b384-a7b7ec3a49c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:54.767587 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-15987c49-9cc5-44ff-b384-a7b7ec3a49c7 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:54.769918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-15987c49-9cc5-44ff-b384-a7b7ec3a49c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:54.777156 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1442/2861] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jun 29 08:14:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:54.789298 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:54.799698 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c918c1d-371e-4831-bb19-d4944301ccaa service swift] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:54.804797 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:54.805457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:54.805737 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:54.806013 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:54.813994 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:54.816011 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:54.824006 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1420/2862] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:14:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:14:55.147035 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e3ed-6b08-47ab-a33b-4759f5b4b5a4 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:55.147329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e3ed-6b08-47ab-a33b-4759f5b4b5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:55.147614 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e3ed-6b08-47ab-a33b-4759f5b4b5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:55.168389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-e218e3ed-6b08-47ab-a33b-4759f5b4b5a4 None None] MFA Rules not processed for user `c813112306f840008f94dec2d2c844f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:55.169648 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e218e3ed-6b08-47ab-a33b-4759f5b4b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:55.177830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1443/2863] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jun 29 08:14:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:55.189803 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:55.200787 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa909a-2711-419d-8ebd-bc2f1efeda76 service swift] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:55.205670 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeaa7c-b33e-43cf-a2c6-351012054901 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a411a835bce0475fae1db6e099dc2605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:55.206651 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeaa7c-b33e-43cf-a2c6-351012054901 service swift] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:55.207111 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeaa7c-b33e-43cf-a2c6-351012054901 service swift] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:55.207415 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeaa7c-b33e-43cf-a2c6-351012054901 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:55.215700 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eeaa7c-b33e-43cf-a2c6-351012054901 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:55.217289 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eeaa7c-b33e-43cf-a2c6-351012054901 service swift] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:55.225833 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1421/2864] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jun 29 08:14:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:14:57.309077 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e3ed-6b08-47ab-a33b-4759f5b4b5a4 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:57.372150 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e3ed-6b08-47ab-a33b-4759f5b4b5a4 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:57.376608 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:57.377506 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:57.377779 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:57.378063 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] PATH_INFO: `/v3/users/0bf77869a8f84d9ca61053e6a0fc08d9` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:57.379239 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf77869a8f84d9ca61053e6a0fc08d9)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:57.381194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:57.430264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf77869a8f84d9ca61053e6a0fc08d9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.441899 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf77869a8f84d9ca61053e6a0fc08d9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.442558 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf77869a8f84d9ca61053e6a0fc08d9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.449989 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf77869a8f84d9ca61053e6a0fc08d9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.450742 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf77869a8f84d9ca61053e6a0fc08d9'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.463550 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1444/2865] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:14:57 2022] DELETE /identity/v3/users/0bf77869a8f84d9ca61053e6a0fc08d9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:57.468144 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:57.468637 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:57.468753 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:57.490176 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:57.491145 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:57.537652 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=100 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:57.552234 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1422/2866] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:57.562109 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:57.563516 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:57.571280 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:57.623512 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb70d9e-6d1f-4baa-a7c7-d473dea497c8 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:57.627157 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d8f0e-7430-4ae0-8f32-d8eb8d95aba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:57.627796 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d8f0e-7430-4ae0-8f32-d8eb8d95aba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:57.628618 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d8f0e-7430-4ae0-8f32-d8eb8d95aba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:57.628904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d8f0e-7430-4ae0-8f32-d8eb8d95aba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:57.638073 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d8f0e-7430-4ae0-8f32-d8eb8d95aba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:57.639505 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d8f0e-7430-4ae0-8f32-d8eb8d95aba9 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:57.649919 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1445/2867] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:14:57.818107 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:57.819657 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:57.832857 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849a5e2-a360-4647-8f9b-ff5478082413 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:57.837621 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:57.838573 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:57.838920 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:57.839826 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] PATH_INFO: `/v3/projects/74c7b1ba386348c0b22d4cc4bf296735` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:57.847020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c7b1ba386348c0b22d4cc4bf296735)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:57.848966 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:57.897421 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c7b1ba386348c0b22d4cc4bf296735', 'request_id': 'req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.903024 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c7b1ba386348c0b22d4cc4bf296735', 'request_id': 'req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.909214 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c7b1ba386348c0b22d4cc4bf296735', 'request_id': 'req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.921999 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] The token cache is being invalidate because project 74c7b1ba386348c0b22d4cc4bf296735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:14:57.922290 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.922930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-7a9b7024-9c1a-4acc-ba37-2cc3cb02fc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:14:57.924457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1423/2868] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:14:57 2022] DELETE /identity/v3/projects/74c7b1ba386348c0b22d4cc4bf296735 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:57.930171 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31172-ea8c-4346-af03-1e0fd4bcd829 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:57.930458 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31172-ea8c-4346-af03-1e0fd4bcd829 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:57.930736 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31172-ea8c-4346-af03-1e0fd4bcd829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:57.948930 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-56a31172-ea8c-4346-af03-1e0fd4bcd829 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:57.950160 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-56a31172-ea8c-4346-af03-1e0fd4bcd829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:58.011016 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1446/2869] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:58.018921 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062598c-3927-4cdc-b75c-f66d21e8a3e3 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.019215 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062598c-3927-4cdc-b75c-f66d21e8a3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.019556 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062598c-3927-4cdc-b75c-f66d21e8a3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.038033 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2062598c-3927-4cdc-b75c-f66d21e8a3e3 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:58.039236 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2062598c-3927-4cdc-b75c-f66d21e8a3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:58.047135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1424/2870] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:58.051970 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a31172-ea8c-4346-af03-1e0fd4bcd829 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:58.062102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a31172-ea8c-4346-af03-1e0fd4bcd829 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:58.067310 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:58.067971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.068254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.068529 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.069281 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80107) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:14:58.069603 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:58.071187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:58.082704 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1447/2871] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jun 29 08:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:14:58.087136 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062598c-3927-4cdc-b75c-f66d21e8a3e3 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:58.096265 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062598c-3927-4cdc-b75c-f66d21e8a3e3 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:58.101166 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:58.101796 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.102083 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.102363 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.103054 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:58.104945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:58.131525 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1425/2872] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:14:58 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:14:58.135699 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:58.145948 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd84c90-b998-4465-b6a6-9991705618f1 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:58.201615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:58.202287 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.202581 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.202863 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.203563 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:58.205588 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:58.225839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1448/2873] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:14:58 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:14:58.229930 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:58.240712 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5d7e6-f2bd-4d1d-b5fa-1fcbfe90d6d9 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:58.245274 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:58.245924 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.246206 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.246517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.247264 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:58.248829 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:58.260150 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1426/2874] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:14:58.264086 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:58.274548 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d5ccc-0630-46c0-9bc3-4af36a5bb278 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:58.279323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccee12-cb55-4b10-8034-5d43171804c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:58.279979 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccee12-cb55-4b10-8034-5d43171804c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.280263 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccee12-cb55-4b10-8034-5d43171804c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.280541 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccee12-cb55-4b10-8034-5d43171804c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.281305 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccee12-cb55-4b10-8034-5d43171804c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:58.282904 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccee12-cb55-4b10-8034-5d43171804c5 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:58.293519 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1449/2875] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:14:58.298020 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:58.310021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3157b-2533-444b-be47-063b4867de40 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:58.316957 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:58.317958 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.318242 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.318522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] PATH_INFO: `/v3/projects/1d0c865636434569b3e25362dc02d5b8/users/36fcd3d9c490413ab9429bccdad444cb/roles/acffa214db594ed9803f8f5376b15d42` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.334840 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0c865636434569b3e25362dc02d5b8, user_id=36fcd3d9c490413ab9429bccdad444cb, role_id=acffa214db594ed9803f8f5376b15d42)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:58.337522 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:58.352171 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1427/2876] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:14:58 2022] PUT /identity/v3/projects/1d0c865636434569b3e25362dc02d5b8/users/36fcd3d9c490413ab9429bccdad444cb/roles/acffa214db594ed9803f8f5376b15d42 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:14:58.357194 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1450c-fe03-4f43-bb66-cf5a5f59ec55 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.357703 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1450c-fe03-4f43-bb66-cf5a5f59ec55 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.357815 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1450c-fe03-4f43-bb66-cf5a5f59ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.377290 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-cad1450c-fe03-4f43-bb66-cf5a5f59ec55 None None] MFA Rules not processed for user `b406de2279df4cff9aec118a80931d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:14:58.379121 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-cad1450c-fe03-4f43-bb66-cf5a5f59ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:58.387763 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:14:58.446000 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1450/2877] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:14:58.457196 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:14:58.460912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:14:58.529253 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b5ebb-97f6-48e9-ab09-d3a21bf163fe admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:14:58.533177 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:14:58.533854 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:14:58.534138 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:14:58.535105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:14:58.545873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:14:58.547335 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:14:58.559734 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1428/2878] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:15:02.571241 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95c64a-ebb5-4218-b2a6-36d57d9c7e2b None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:02.571726 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95c64a-ebb5-4218-b2a6-36d57d9c7e2b None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:02.571839 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95c64a-ebb5-4218-b2a6-36d57d9c7e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:02.610912 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-2f95c64a-ebb5-4218-b2a6-36d57d9c7e2b None None] MFA Rules not processed for user `36fcd3d9c490413ab9429bccdad444cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:15:02.612484 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-2f95c64a-ebb5-4218-b2a6-36d57d9c7e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:15:02.736897 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1451/2879] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:15:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:15:02.741630 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:02.758372 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:15:02.810417 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e1352-1297-4b02-ac4e-53ff754109bb service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:02.815286 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:02.816500 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:02.816780 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:02.817081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] PATH_INFO: `/v3/projects` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:02.818583 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:02.820510 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:02.862820 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1429/2880] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jun 29 08:15:02 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:15:02.867395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95c64a-ebb5-4218-b2a6-36d57d9c7e2b None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:02.878956 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95c64a-ebb5-4218-b2a6-36d57d9c7e2b None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:02.939254 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:02.939831 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:02.940066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:02.940347 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] PATH_INFO: `/v3/users` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:02.941708 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:02.943709 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:02.966898 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1452/2881] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jun 29 08:15:02 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:15:02.970726 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:02.981680 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc936989-a0c4-405b-908c-541c832fc674 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:02.986341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:02.987003 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:02.987382 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:02.987560 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:02.988293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:02.989945 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:03.009941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1430/2882] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:15:03.012143 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:03.027464 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd512d5-4a4c-452f-9f56-1fdb9879d71c admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:03.032412 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:03.033784 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:03.034152 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:03.035346 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] PATH_INFO: `/v3/projects/735bc0d6dec94483ba7924dd90462a56/users/03abe7389e3a4db1b163ebd81e1b251b/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:03.038520 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:15:03.058434 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] RBAC: Authorizing `identity:create_grant(project_id=735bc0d6dec94483ba7924dd90462a56, user_id=03abe7389e3a4db1b163ebd81e1b251b, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:03.061323 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:03.080106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1453/2883] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:15:03 2022] PUT /identity/v3/projects/735bc0d6dec94483ba7924dd90462a56/users/03abe7389e3a4db1b163ebd81e1b251b/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:15:03.084532 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:03.095105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dced31-d71d-4b09-9f8a-adb8c1bf1e43 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:03.155729 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c036ca-f009-4d14-85ea-14f90487fe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:03.156487 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c036ca-f009-4d14-85ea-14f90487fe28 admin admin] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:03.156771 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c036ca-f009-4d14-85ea-14f90487fe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:03.157081 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c036ca-f009-4d14-85ea-14f90487fe28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:03.157810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c036ca-f009-4d14-85ea-14f90487fe28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:03.159553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c036ca-f009-4d14-85ea-14f90487fe28 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:03.169225 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1431/2884] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jun 29 08:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:15:03.173423 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:03.185886 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8fa91-d020-4c78-951c-61564838252d admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:03.192054 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:03.193027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:03.193395 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:03.193750 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] PATH_INFO: `/v3/projects/735bc0d6dec94483ba7924dd90462a56/users/03abe7389e3a4db1b163ebd81e1b251b/roles/fe0d00714612495091430762d114121a` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:03.195889 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=735bc0d6dec94483ba7924dd90462a56, user_id=03abe7389e3a4db1b163ebd81e1b251b, role_id=fe0d00714612495091430762d114121a)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:03.198634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:03.213377 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1454/2885] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jun 29 08:15:03 2022] PUT /identity/v3/projects/735bc0d6dec94483ba7924dd90462a56/users/03abe7389e3a4db1b163ebd81e1b251b/roles/fe0d00714612495091430762d114121a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:15:07.857284 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c0504-1cbb-4eb7-ba7b-9de6979f1270 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:07.857873 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c0504-1cbb-4eb7-ba7b-9de6979f1270 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:07.857979 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c0504-1cbb-4eb7-ba7b-9de6979f1270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:07.892665 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-8d4c0504-1cbb-4eb7-ba7b-9de6979f1270 None None] MFA Rules not processed for user `03abe7389e3a4db1b163ebd81e1b251b`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:15:07.893987 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-8d4c0504-1cbb-4eb7-ba7b-9de6979f1270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:15:07.982019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1432/2886] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:15:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:15:07.998261 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:08.001170 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:15:08.101893 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95887cbe-c80f-4b49-914a-6cdde6a2a052 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:08.111329 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:08.112012 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:08.112424 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:08.112866 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:08.178634 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:08.180200 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:08.190234 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1455/2887] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:15:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:15:17.977998 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:15:18.185701 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=75,keystone:INSERT=2 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:15:23.407036 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc8fe6-fe84-4ef7-ae3f-c481303fc7e5 None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:23.407561 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc8fe6-fe84-4ef7-ae3f-c481303fc7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:23.407664 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc8fe6-fe84-4ef7-ae3f-c481303fc7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:23.424845 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-e6cc8fe6-fe84-4ef7-ae3f-c481303fc7e5 None None] MFA Rules not processed for user `36fcd3d9c490413ab9429bccdad444cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:15:23.426422 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-e6cc8fe6-fe84-4ef7-ae3f-c481303fc7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:15:23.433564 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1433/2888] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:15:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:15:23.754580 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:23.756104 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:15:23.814226 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b29eb-eae8-442a-af3a-bd6a6edb6d77 service nova] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:23.817797 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:23.818435 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:23.818724 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:23.819264 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:23.821106 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:15:23.830801 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:23.832187 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:23.840755 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1456/2889] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:15:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:15:33.419739 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=3 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:15:33.837199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=26 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:15:38.926066 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc8fe6-fe84-4ef7-ae3f-c481303fc7e5 None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:38.937056 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc8fe6-fe84-4ef7-ae3f-c481303fc7e5 None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:38.940531 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:38.941271 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:38.941553 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:38.941824 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:38.952341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:38.953936 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:38.963362 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1434/2890] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:15:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:15:40.528245 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:15:40.560773 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc5f2-27f0-42e2-b8b5-cf498ac22302 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:15:40.566635 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ca42d-77c5-494b-8821-35e05d23679b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:15:40.567316 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ca42d-77c5-494b-8821-35e05d23679b service nova] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:15:40.567712 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ca42d-77c5-494b-8821-35e05d23679b service nova] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:15:40.568018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ca42d-77c5-494b-8821-35e05d23679b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:15:40.570018 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-425ca42d-77c5-494b-8821-35e05d23679b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:15:40.597283 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ca42d-77c5-494b-8821-35e05d23679b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:15:40.599024 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ca42d-77c5-494b-8821-35e05d23679b service nova] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:15:40.613105 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1457/2891] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:15:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:15:48.959398 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=6 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:15:50.607177 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=18 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:17:01.732606 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:17:01.734720 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:17:01.838202 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a590c0-c8ec-4a9e-a3ea-9aeb01d3859b service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:17:01.844450 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d2751-275d-42b3-a9e5-7fa551ac5668 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:17:01.845727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2751-275d-42b3-a9e5-7fa551ac5668 service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:17:01.845727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2751-275d-42b3-a9e5-7fa551ac5668 service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:17:01.845727 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2751-275d-42b3-a9e5-7fa551ac5668 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:17:01.847483 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-712d2751-275d-42b3-a9e5-7fa551ac5668 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:17:01.860823 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d2751-275d-42b3-a9e5-7fa551ac5668 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:17:01.863750 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d2751-275d-42b3-a9e5-7fa551ac5668 service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:17:01.875517 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1435/2892] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:17:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:17:11.869923 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=41 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:17:38.464554 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] REQUEST_METHOD: `POST` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:17:38.464961 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:17:38.465237 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:17:38.483260 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.auth.core [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] MFA Rules not processed for user `36fcd3d9c490413ab9429bccdad444cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80107) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:17:38.485389 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:17:38.493092 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1458/2893] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:17:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:17:48.478188 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=3 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:19:29.845285 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a681434-3184-4355-a93e-46ba8649c97d None None] REQUEST_METHOD: `POST` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:19:29.845862 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a681434-3184-4355-a93e-46ba8649c97d None None] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:19:29.845989 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a681434-3184-4355-a93e-46ba8649c97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:19:29.864511 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.auth.core [None req-2a681434-3184-4355-a93e-46ba8649c97d None None] MFA Rules not processed for user `36fcd3d9c490413ab9429bccdad444cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80108) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:19:29.865735 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2a681434-3184-4355-a93e-46ba8649c97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:19:29.874156 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1436/2894] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jun 29 08:19:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:19:29.915255 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:19:29.916764 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:19:29.998366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3906d6-5660-4c72-bdaa-05794b248b51 None None] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:19:30.001867 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24630ccd20794f19a138a19187348920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:19:30.003102 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:19:30.003380 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:19:30.003652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:19:30.005432 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:19:30.019720 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:19:30.021165 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:19:30.032066 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1459/2895] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:19:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:19:30.337131 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a681434-3184-4355-a93e-46ba8649c97d None None] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:19:30.356912 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a681434-3184-4355-a93e-46ba8649c97d None None] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:19:30.360427 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:19:30.361095 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:19:30.361380 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:19:30.361659 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:19:30.363911 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:19:30.386088 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:19:30.387505 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:19:30.395410 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1437/2896] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:19:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:19:30.865652 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:19:30.875644 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044ff7d-66dc-4340-af07-fbe47926e2d4 service cinder] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:19:30.879566 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:19:30.880208 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:19:30.880496 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:19:30.880792 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:19:30.882693 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:19:30.899067 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:19:30.900504 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:19:30.913444 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1460/2897] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:19:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:19:40.391848 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=18 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:19:40.907673 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=41 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:19:43.732191 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:19:43.741092 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed05c4-0aa1-47bc-b127-019be622c556 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:19:43.744874 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:19:43.745575 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:19:43.745859 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:19:43.746135 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:19:43.747932 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:19:43.758594 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:19:43.760047 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:19:43.770151 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1438/2898] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:19:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:19:53.765386 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=6 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:15.488747 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:15.497746 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b3eb-f6dc-45eb-a584-e2e1f273e3b5 service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:15.501366 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:15.502002 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:15.502294 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:15.502573 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:15.510793 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:15.512223 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:15.521029 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1461/2899] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:21:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:21:16.388283 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:16.403711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7dcf-f051-479d-ab78-c6b811ba8f14 service neutron] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:16.409458 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a267b6f64f5f49b8b3f5837e701bc216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:16.410180 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:16.410559 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:16.411019 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:16.412877 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:16.426770 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:16.428498 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:16.440240 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1439/2900] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:21:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:21:25.517082 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=6 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:26.435941 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=6 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:28.859607 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:28.869308 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0646182-fcf4-48bc-a7d7-68666e157fca service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:28.873120 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45a558f9c7c84a05a90595fda7fccd40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:28.873706 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:28.873997 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:28.874276 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:28.933981 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.935407 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.945437 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1462/2901] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:21:32.041653 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:32.053002 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415111-05a3-481b-8677-d1a217ab3c86 service nova] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:32.058861 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:32.059524 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] REQUEST_METHOD: `GET` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.059810 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.060101 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.061954 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:32.075001 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:32.076618 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:32.091293 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1440/2902] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:21:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:21:35.235832 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.247007 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1732e85-076a-4154-9fb8-43736792de6e service neutron] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.251851 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.252721 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.253074 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.253360 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] PATH_INFO: `/v3/users/36fcd3d9c490413ab9429bccdad444cb` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.254561 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36fcd3d9c490413ab9429bccdad444cb)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.256624 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.303530 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36fcd3d9c490413ab9429bccdad444cb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.316249 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36fcd3d9c490413ab9429bccdad444cb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.316926 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36fcd3d9c490413ab9429bccdad444cb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.325219 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36fcd3d9c490413ab9429bccdad444cb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.325830 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36fcd3d9c490413ab9429bccdad444cb'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.338023 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1463/2903] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:21:35 2022] DELETE /identity/v3/users/36fcd3d9c490413ab9429bccdad444cb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:35.509021 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.519863 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827f27d-b8c8-4c20-9d20-d2077ffe08ec service placement] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.577600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.578457 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.578754 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.579032 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] PATH_INFO: `/v3/projects/1d0c865636434569b3e25362dc02d5b8` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.580203 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0c865636434569b3e25362dc02d5b8)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.582224 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.628134 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0c865636434569b3e25362dc02d5b8', 'request_id': 'req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.633546 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0c865636434569b3e25362dc02d5b8', 'request_id': 'req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.639542 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0c865636434569b3e25362dc02d5b8', 'request_id': 'req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.651821 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] The token cache is being invalidate because project 1d0c865636434569b3e25362dc02d5b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:21:35.652105 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.652711 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.654232 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1441/2904] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:21:35 2022] DELETE /identity/v3/projects/1d0c865636434569b3e25362dc02d5b8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:35.658179 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.659615 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:35.673262 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102da147-bb3c-467e-b148-189e794cd3e7 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.730428 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.731299 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.731586 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.731864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] PATH_INFO: `/v3/users/03abe7389e3a4db1b163ebd81e1b251b` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.733090 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03abe7389e3a4db1b163ebd81e1b251b)` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.735028 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.740256 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:35.779713 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03abe7389e3a4db1b163ebd81e1b251b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.791907 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03abe7389e3a4db1b163ebd81e1b251b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.792560 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03abe7389e3a4db1b163ebd81e1b251b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.799540 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03abe7389e3a4db1b163ebd81e1b251b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.800199 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.notifications [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03abe7389e3a4db1b163ebd81e1b251b'} {{(pid=80107) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:35.812888 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1464/2905] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jun 29 08:21:35 2022] DELETE /identity/v3/users/03abe7389e3a4db1b163ebd81e1b251b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:35.985367 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Authenticating user token {{(pid=80108) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.986894 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.fernet_utils [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80108) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:36.005598 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6f6f8-d9bd-4f20-ae54-c9dfef8d3355 admin admin] Validating token access rules against request {{(pid=80108) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:36.064160 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b406de2279df4cff9aec118a80931d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e7f90bb9da44679fd899e1c8b90999', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80108) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:36.065048 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:36.065330 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:36.065600 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] PATH_INFO: `/v3/projects/735bc0d6dec94483ba7924dd90462a56` {{(pid=80108) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:36.066775 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=735bc0d6dec94483ba7924dd90462a56)` {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:36.068692 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] RBAC: Authorization granted {{(pid=80108) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:36.089329 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:36.116721 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735bc0d6dec94483ba7924dd90462a56', 'request_id': 'req-8a83095e-0948-47ac-8089-1038245154aa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:36.122341 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735bc0d6dec94483ba7924dd90462a56', 'request_id': 'req-8a83095e-0948-47ac-8089-1038245154aa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:36.128404 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735bc0d6dec94483ba7924dd90462a56', 'request_id': 'req-8a83095e-0948-47ac-8089-1038245154aa'} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:36.141369 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] The token cache is being invalidate because project 735bc0d6dec94483ba7924dd90462a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80108) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:21:36.141657 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:36.142268 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG keystone.notifications [None req-8a83095e-0948-47ac-8089-1038245154aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80108) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:21:36.143776 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: [pid: 80108|app: 0|req: 1442/2906] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jun 29 08:21:35 2022] DELETE /identity/v3/projects/735bc0d6dec94483ba7924dd90462a56 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:45.809393 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:46.138399 dv-cinder-768105-14-iscsi devstack@keystone.service[80108]: DEBUG dbcounter [-] [80108] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:04.689452 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Authenticating user token {{(pid=80107) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:04.690864 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:04.760178 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d6c9c-2462-4ad7-b620-0b6f4f774697 admin admin] Validating token access rules against request {{(pid=80107) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:04.819971 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91f4c8-4321-41e7-a4d8-8d0531ac63a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fdc728b26894afa85073c08c7c34ecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de99a06049e847098fd35e509e1c797a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80107) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:04.820633 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91f4c8-4321-41e7-a4d8-8d0531ac63a0 service placement] REQUEST_METHOD: `GET` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.820918 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91f4c8-4321-41e7-a4d8-8d0531ac63a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.821233 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91f4c8-4321-41e7-a4d8-8d0531ac63a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80107) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.823037 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.fernet_utils [None req-9c91f4c8-4321-41e7-a4d8-8d0531ac63a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80107) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:04.835027 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91f4c8-4321-41e7-a4d8-8d0531ac63a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:04.836612 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91f4c8-4321-41e7-a4d8-8d0531ac63a0 service placement] RBAC: Authorization granted {{(pid=80107) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:04.846489 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: [pid: 80107|app: 0|req: 1465/2907] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jun 29 08:22:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:22:14.842238 dv-cinder-768105-14-iscsi devstack@keystone.service[80107]: DEBUG dbcounter [-] [80107] Writing DB stats keystone:SELECT=58 {{(pid=80107) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}